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Introduction 


The  Acting  Technical  Director  encourages  scientists  and  engineers  at  the  Navy  Personnel 
Research  and  Development  Center  (NPRDC)  to  generate  new  and  innovative  proposals  to  promote 
scientific  and  technological  growth  in  the  organization  and  the  development  of  knowledge  and 
technology  of  interest  to  the  Navy.  Support  for  this  is  provided  by  discretionary  funding  furnished 
by  the  Independent  Research  (IR)  and  Independent  Exploratory  Development  (IED)  programs  of 
the  Office  of  Naval  Research  and  the  Office  of  Naval  Technology.  These  programs  support  initial 
research  and  development  of  interest  to  the  Navy  with  emphasis  on  the  NPRDC  mission  areas  of 
the  acquisition,  training,  and  effective  utilization  of  personnel. 

Funds  are  provided  to  the  Technical  Directors  of  Navy  Laboratories  to  support  innovative  and 
promising  research  and  development  outside  the  procedures  required  under  normal  funding 
authorization.  The  funds  are  to  encourage  creative  efforts  important  to  mission  accomplishment. 
They  enable  promising  researchers  to  spend  a  portion  of  their  time  on  examining  the  feasibility  of 
self-generated  new  ideas  and  scientific  advances.  They  can  provide  important  and  rapid  test  of 
promising  new  technology  and  can  help  fill  gaps  in  the  research  and  development  program.  This 
may  involve  preliminary'  work  on  speculative  solutions  too  risky  to  be  funded  from  existing 
programs.  ■>  v 

The  funds  also  serve  as  means  to  maintain  and  increase  the  necessary  technology  base  skill 
levels  and  build  in-house  expertise  in  areas  likely  to  become  important  in  the  future.  These 
programs  contribute  to  the  scientific  base  for  future  improvements  in  the  manpower,  personnel,  and 
training  system  technology  and  provide  coupling  to  university  and  industrial  research 
communities. 

The  FY89  IR/iED  programs  began  with  a  call  for  proposals  in  June  1988.  Technical  reviews 
were  provided  by  supervisors  and  scientific  consultants  and  six  IR  and  four  IED  projects  were 
funded.  This  report  documents  the  results  and  accomplishments  of  these  projects.  Dr.  W.  F.. 
Montague  administers  the  IR  and  IED  programs,  coordinating  project  selection,  reporting,  and 
reviewing  to  assure  an  innovative  and  productive  program  of  science  and  technology. 

Tables  1  and  2  list  the  projects  active  during  FY89  and  those  supported  in  FY90.  Two  papeis, 
one  IR  and  one  IED,  chosen  by  the  Technical  Director  as  “Best  Papers  of  1989”  are  presented. 
Subsequent  pages  contain  brief  reports  of  research  progress  during  FY89  w'ritien  by  the  principal 
investigators  of  each  project.  Appendix  A  lists  the  IR  and  IED  projects  that  may  have  transitioned 
intoother  projects  or  into  use  by  the  Navy  during  the  year.  Appendix  B  itemizes  the  presentations 
and  publications  from  IR  and  IED  supported  projects.  Appendix  C  presents  awards  and  honors 
related  to  the  projects. 
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Table  1 

Independent  Research 
Work  Units  for  FY89  and  FY90 
(PE  0601152N) 


FY 

Funding 

Telephone  ( K) 

Work  Principal  Internal  (619)  55 - 

Unit  Title  Investigator  Code  or  AV  55  89  90“ 


0601152N.R0001.01 

Brain  activity  during 
visual  recognition 

060 1 1 52N  .R000 1 .02 

Event-related  potential 
correlates  of  memory 
performance 

06O1152N.ROO0i.03 

Using  diagrams  for 
learning  procedural 
tasks 

060 1152N.ROOO  1.04 

Analysis  of  tutoring 
in  technical  training 

060 1 1 52N.ROOO  1 .05 

Stabilization  of 
performance  on  a 
complex  cognitive 
task 

0601152N.R0001.06C 

Experienced-based 
career  development-II 

0601152N.R0001.07C 

Application  of 
machine  intelligence 

0601152N.R0001.08 

Instructional  analysis 

0601 152N.ROOO  1.09 

The  role  of  feedback 
in  computer-based 
training 

0601 152N.ROOO  1.10 

An  exploratory 
examination  of  neural 
networks  as  an  alternative 
to  regression 

0601152N.R0001.il 

Neural  network 

Ryan-Jones, 

Lewis, 

Trejo, 

Hemmer 

14 

37710 

37711 
37711 
37711 

60.0 

26.0 

Williams 

15 

37705 

60.0° 

21.0 

Vogt 

15 

37788 

400 

40.0 

Ellis, 

Montague 

14 

39273 

37849 

65.0 

65.0 

Federico 

14 

37777 

12.5 

0.0 

Mom  son 

12 

39256 

45.0 

17.0 

Sorensen 

15 

37782 

45.0 

0.0 

Gollub 

15 

37849 

12.5 

0.0 

Cowen 

14 

37698 

0.0 

50.0 

Wilkins 

13 

37618 

0.0 

30.0 

Gollub 

15 

37849 

0.0 

:5.0 

•January  1990, 75  percent  of  funds  received. 

Additional  S40K  obtained  from  Office  of  Naval  Research. 
cTransitioned. 
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Table  2 

Independent  Exploratory  Development 
Work  Units  for  FY89  and  FY90 
(PE  0602936N) 


Work 

Unit 

Principal 

Investigator 

Internal 

Code 

Telephone 
(619)  55 
or  AV  55 

FY 

Funding 

(K) 

Tide 

89 

90* 

0602936N.RV36I27.01 

Optimization  of 
nonlinear  objectives 

Krass 

11 

37895 

50.0 

.0 

0602936N.RV36I27.05b 

Empirical  Bayes 

Boyle 

11 

38025 

50.0 

.0 

0602936N.RV36I27.06 

Military  recruit  quality 
and  the  minimum  wage 

Nakada 

15 

39268 

60.0 

.0 

0602936N.RV36127.07 

Application  of  neural 
net  technology  to 
personnel  loss  forecasting 

Huntley 

11 

38038 

35.0 

.0 

0602936N.RV36I27.08b 

Developing  measures  of 
effectiveness  for 
knowledge  discovery 
systems 

Sorensen 

12 

37782 

.0 

50.0 

0602936N.RV36I27.10 

Decomposition  methods 

Thompson 

Krass 

11 

37925 

37895 

50.0 

75.0 

‘January  1990, 75  percent  of  funds  received, 
transitioned. 
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Best  Papers 
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Biography 


David  L.  Ryan-Jones  is  a  Personnel  Research 
Psychologist  in  the  Neurosciences  Division  at  NPRDC.  He 
received  a  B.S.  (1973)  and  M.S.  (1975)  in  Experimental 
Psychology  from  the  University  of  Texas,  F.I  Paso,  and  a  PhD. 
(1988)  from  Florida  State  University  in  Experimental 
Psychology  (Psychobiology).  Dr.  Ryan-Jones  is  a  member  of  the 
American  Psychological  Association,  Human  Factors  Society, 
and  the  Association  for  Research  in  Vision  and  Ophthalmology. 
His  research  interests  are  in  the  electrophysiological  correlates 
of  perceptual  and  cognitive  information  processing. 


Gregory  W.  Lewis  was  born,  raised,  and  educated 
in  the  state  of  Washington.  During  his  graduate  work  at 
Washington,  he  had  extensive  training  in  vision  electrophysi¬ 
ology  and  neurophysiology.  His  doctoral  dissertation  was  in 
the  area  of  vision  biometry  using  ophthalmic  ultrasonography. 
From  1970  to  1974,  Dr.  Lewis  fulfilled  his  military'  obligation 
as  an  Army  officer  in  the  U.S.  Army  Medical  Research 
Laboratory',  Fort  Knox,  Kentucky.  He  has  been  with  NPRDC 
since  1974.  He  developed  and  currently  heads  the  NPRDC 
Neurosciences  Division,  Training  Systems  Department.  His 
research  interests  include  the  psychophysiology  of  individual 
differences,  digital  processing  of  biological  signals,  and 
physiological  correlates  of  brain  and  behavior. 
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Biography 


Leonard  J.  Trejo  was  bom  in  Mexico  City,  Mexico 
in  1955.  He  performed  undergraduate  studies  at  Lock  Haven 
State  College  in  Pennsylvania  and  in  1977,  he  received  his  B.S. 
in  psychology  at  the  University  of  Oregon.  He  performed 
graduate  work  in  psychobiology  at  the  University  of  Michigan 
and  in  psychology  at  the  University  of  California,  San  Diego, 
with  emphasis  on  sensation  and  perception.  He  received  the 
M.A.  (1980)  and  PhD.  (1982)  from  the  University  of  California, 
San  Diego  for  research  on  the  neurophysiology  of  the  pupillary' 
light  reflex  and  visual  sensitivity  loss  in  hereditary  retinal 
degeneration.  From  1982  to  1984,  he  served  as  Senior  Fellow  in 
the  Department  of  Ophthalmology  at  the  University  of 
Washington,  where  he  performed  research  on  retinal  toxicity,  physiology  of  color  vision,  and 
neuroanatomy  of  the  visual  cortex.  Since  September  1984,  he  has  been  a  Personnel  Research 
Psychologist  at  NPRDC.  His  research  interests  include  sensation  and  perception,  color  vision, 
visual  pathways,  cognition,  and  human  performance. 


Joan  D.  Hemmer  is  a  Professor  in  the  Psychology 
Department,  St.  Cloud  State  University,  St.  Cloud,  Minnesota. 
She  teaches  psychology  statistics,  research  methods,  psychology 
of  learning,  personality  theories,  and  general  psychology.  She 
earned  her  B.A.  at  the  University  of  Minnesota  and  the  M.A.  and 
PhD.  degrees  at  the  University  of  Colorado  at  Boulder  (1972). 
She  taught  psychology  at  the  University  of  Massachusetts  in 
Amherst,  from  1971-1976,  prior  to  her  move  to  Minnesota,  her 
home  state.  In  1984-1985,  she  trained  in  psychophysiology  at 
the  University  ot  Minnesota  Medical  School  in  Minneapolis  in 
Psychiatry  Research.  She  was  an  Office  of  Naval  Research/ 
American  Society  of  Engineering  Education  Summer  Facility 


Fellow  at  the  Neurosciences  Division,  NPRDC,  during  the  summers  of  1988  and  1989.  Her 


research  interests  are  in  the  electrophysiological  correlates  of  learning  and  performance  of 
complex  visual  tasks. 
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Brain  Activity  During  Visual  Recognition 

David  L.  Ryan- Jones 
Gregory  \V.  Lewis 
Leonard  J.  Trejo 
Joan  D.  Hemmer 


Navy  combat  systems  have  become  increasingly  complex  over  the  last  few 
decades.  There  is  a  need  to  understand  the  cognitive  demands  placed  upon  the 
operators  of  these  systems.  One  technique  that  can  be  used  to  estimate  the  cognitive 
demands  of  a  task  is  to  record  the  event-related  electrical  potentials  (ERPs) 
generated  by  the  brain  during  performance  of  the  task.  In  this  study,  we  examined 
the  spatial  and  temporal  distribution  ofERP  components  during  a  facial  recognition 
task.  We  found  that  familiar  faces  elicited  significantly  different  ERP  activity  than 
unfamiliar  faces.  Although  more  research  will  be  required,  the  results  suggest  this 
experimental  technique  may  directly  index  the  degree  to  which  an  individual  is 
familiar  with  or  has  knowledge  about  a  visual  target.  Potential  applications  may  be 
in  the  areas  of  educational  and  training  evaluation,  personnel  security ’  and 
reliability'  assessment,  and  selection  and  classification. 


Introduction 

Navy  combat  systems  have  become 
increasingly  complex  over  the  last  several 
decades.  Many  system  operators  depend 
highly  upon  complex  visual  and  auditory 
cueing  during  the  performance  of  their 
missions.  For  example,  radar  target 
acquisition  systems  no  longer  present  only 
simple  blips,  but  instead  present  very 
complex  computer-generated  symbols  that 
cue  for  the  important  characteristics  of  the 
target,  such  as  speed,  range,  altitude,  and 
direction.  These  complex  alphanumeric  and 
graphical  data  presentation  techniques  may 
make  it  increasingly  difficult  for  an  operator 
to  assimilate  and  interpret  critical  battlefield 
information  under  operational  time 
constraints  (Chechile,  Eggleston,  Fleishman, 
&  Sasseville,  1989;  Mitchell  &  Miller,  1983). 
Thus,  the  personnel  operating  these  systems 


may  require  cognitive  skills  and  abilities  that 
were  not  required  several  years  ago.  In 
contrast  to  the  increasing  complexity  of 
operational  systems,  traditional  selection  and 
classification  tests  have  not  yet  shown  greatly 
improved  predictive  validity.  These  tests  are 
able  to  adequately  predict  training  or 
academic  performance,  but  not  on-the-job 
performance  (Ghiselli,  1966;  Vineberg  & 
Joyner,  1982). 

There  is  a  need  to  impiove  the  ability  to 
assess  the  unique  qualities  of  individuals, 
including  their  underlying  perceptual  and 
cognitive  processes,  and  to  use  this  improved 
assessment  in  Navy  education  and  training 
programs  to  ensure  that  these  individuals 
successfully  perform  their  jobs  once  they  are 
assigned  to  operational  units.  One  factor  that 
underlies  all  techniques  used  for  personnel 
selection,  classification,  and  training,  as  well 
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as  on-the-job  performance,  is  the  perceptual 
and  cognitive  processing  in  the  cerebral 
cortex  of  the  brain.  Neuroelectric  technology, 
which  includes  the  event-related  brain 
potential  (ERP),  is  currently  one  of  the  best 
methods  to  directly  assess  neural  processing 
in  individuals.  The  spatial  and  temporal 
sequence  of  cortical  processing  is  preserved 
in  the  ERP  record,  from  early  sense; y 
processing  to  complex  decision-making 
processes  to  motor  output,  regardless  of 
whether  ERP  records  are  analyzed  as  single 
epochs  or  as  signal  averages.  Thus,  the  ERP 
is  a  valuable  technique  for  studying  these 
processes. 

Objective 

The  general  objectives  of  this  research 
were  to  develop  an  understanding  of  the 
neural  processes  that  subserve  visual 
recognition,  and  to  develop  a  reliable  method 
for  predicting  which  individuals  will  perform 
best  in  jobs  in  which  visual  recognition  is  a 
critical  or  substantive  part  of  the  job.  The 
specific  objective  of  the  research  reported 
here  was  to  assess  the  spatial  and  temporal 
sequence  of  cortical  processing  initiated  by 
viewing  complex  visual  images.  In  particular, 
we  were  interested  in  the  kinds  of 
“real-world”  visual  images  that  Navy 
personnel  might  encounter  during  the 
performance  of  their  duties.  Job-related 
images  could  include  such  things  as  scenery, 
ships,  airplanes,  radar  symbology,  or  human 
faces.  We  selected  human  faces  as  the  initial 
stimuli  for  this  study  for  two  reasons.  First, 
previous  research  has  suggested  that  facial 
recognition  tasks  can  tap  a  wide  variety  of 
cortical  processes  involved  in  recognition  in 
general,  from  image  segmentation  to  memory 
retrieval  to  decision  making.  There  is  some 
reason  to  believe  that  the  neural  processes 
underlying  visual  recogni'ion  are  relatively 
independent  of  the  natuiv  of  the  stimulus. 
Second,  there  is  a  substantial  body  of 


research  literature  on  the  behavioral  aspects 
of  recognition,  as  well  as  the 
neuroanatomical  substrates  of  facial 
recognition,  that  allows  us  to  identify  the 
important  experimental  variables  that  affect 
recognition.  Thus,  it  is  possible  to 
immediately  begin  the  research  with  an 
advanced  understanding  of  the  task. 

Background 

Facial  Recognition 

The  human  ability  to  remember  the  face 
of  a  previously  known  individual  is 
remarkable  in  terms  of  speed  and  accuracy. 
Recognition  accuracy  has  been  shown  to  be 
relatively  stable  over  an  unreinforced 
retention  interval  as  long  as  15  years 
(Bahrick,  Bahrick,  &  Wittlinger,  1975).  The 
ability  to  remember  faces  is  affected  by  a 
wide  variety  of  variables,  including 
chronological  age  (Diesfeldt  &  Vink,  1989), 
amount  of  rehearsal  (Proctor,  1985),  context 
of  the  recognition  task  (Tiberghien,  1986), 
circumstances  of  the  original  encounter 
(Brown,  Deffenbacher,  &  Sturgill,  1977), 
depth  of  processing  (Bower  &  Karlin,  1974), 
complexity  of  the  facial  features  (Goldstein 
&  Chance,  1974),  viewing  duration  (Loftus, 
Nelson,  &  Kallman,  1983;  Smith  &  Neilsen, 
1970),  and  image  quality  (Valentine,  1988). 
Studies  have  also  suggested  that  there  may  be 
a  high  degree  of  hemispheric  specialization 
in  the  processing  of  facial  information 
(Young,  1984).  Presentation  of  faces  in  the 
left  half  of  the  visual  field  (information 
processed  in  the  right  hemisphere)  results  in 
better  recognition  and  shorter  reaction  time 
than  presentation  of  faces  in  the  right  half  of 
the  visual  field  (information  processed  in  the 
left  hemisphere).  Facial  recognition  has,  at 
times,  been  explained  in  terms  of  a  general 
memory  processor  that  is  independent  of  the 
nature  of  the  material  to  be  remembered 
(Paivio,  1971),  a  dedicated  face  processor 
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(Diamond  &  Carey,  1986;  Perrett,  Mistlin,  & 
Chitty,  i987),  and  a  general  object  processor 
that  is  independent  of  the  nature  of  the  objects 
(Cohen,  1973;  Hay  &  Young,  1982).  One 
widely  accepted  theory  is  that  there  are 
several  different,  but  coexisting,  systems  for 
processing  information  held  in  memory  in  the 
human  brain  (Tulving,  1985),  but  it  has  not 
been  established  that  the  neural  mechanisms 
underlying  facial  recognition  are  uniquely 
different  than  the  neural  mechanisms  for  the 
recognition  of  other  objects. 

Much  of  what  we  know  about  the 
neurophysiology  of  facial  recognition  comes 
from  the  study  of  those  individuals  afflicted 
with  a  disorder  called  prosopagnosia,  and 
from  studies  of  the  responses  to  facial  stimuli 
by  single  conical  cells  in  primates. 
Prosopagnosia  is  the  inability  to  recognize 
familiar  faces,  although  individuals  with  this 
deficit  are  generally  unable  to  recognize  other 
classes  of  objects  as  well  (Bauer,  1986). 
These  deficits  are  seen  in  patients  who  have 
cortical  lesions  near  the  ventromedial 
boundary  of  the  temporal  and  occipital  lobes 
of  the  brain  (Damasio,  Damasio,  &  Tranel, 
1986).  These  lesions  have  been  confirmed  by 
computerized  axial  tomography  and 
magnetic  resonance  imaging.  Except  in  very 
severe  cases,  patients  with  these  types  of 
lesions  do  not  exhibit  a  complete  failure  to 
recognize  familiar  faces.  Instead,  there  is 
most  often  only  a  slight  to  moderate  decrease 
in  recognition  accuracy,  and  a  corresponding 
increase  in  the  time  to  make  a  response.  Even 
though  the  patient  may  deny  tecognizing  a 
face,  the  patient  may  feel  that  the  face  is  or 
should  be  familiar.  These  emotional 
responses,  or  feelings  of  familiarity,  are 
believed  to  result  from  information  that 
travels  to  the  limbic  system  by  way  of 
undamaged  pathways  in  the  dorso-medi'’’ 
part  of  the  parietal  lobe.  The  exact  del. . 
seen  in  prosopagnosia  depend  upon  the  k 
of  the  lesion,  whether  the  lesion  is  bilateral  ... 


unilateral,  and  the  amount  of  day-to-day 
contact  the  patient  has  with  the  individuals  to 
be  recognized.  In  patients  with  unilateral 
lesions,  the  deficit  is  more  severe  if  the  lesion 
is  located  in  the  right  hemisphere,  supporting 
the  behavioral  finding  of  a  right  hemispheric 
advantage  in  recognition  accuracy  (De  Haan 
&  Hay,  1986).  The  deficit  seems  most  severe 
in  patients  with  bilateral  lesions  of  the 
occipital-temporal  region.  Studies  have 
shown  that  when  these  patients  deny 
recognizing  a  familiar  individual  when 
presented  with  the  familiar  face,  they  still 
generate  physiological  responses,  such  as 
galvanic  skin  responses,  that  are  similar  to 
those  generated  by  faces  they  do  recognize. 
This  suggests  that  the  underlying  perceptual 
mechanism  of  recognition  proceeds 
automatically,  and  that  the  deficit  is  one 
involving  the  cognitive  evaluation  of  the 
perceptual  information  (Tranel  &  Damasio, 
1985).  Thus,  prosopagnosia  is  best  viewed  as 
a  memory  deficit  rather  than  a  perceptual 
deficit. 

There  have  been  several  studies  in 
recent  years  that  have  examined  the 
responses  to  facial  stimuli  by  cortical  neurons 
in  the  superior  temporal  sulcus  of  the 
temporal  lobe  of  the  primate  brain.  These 
cells  have  sometimes  been  classified  in  terms 
of  the  aspects  of  the  face  to  which  they  best 
respond,  such  as  tue  full  face,  profile,  back  of 
head,  head  up,  and  head  down  (Perrett, 
Mistlin,  Potter,  Smith,  Head,  Chitty, 
Broennimann,  Milner,  &  Jeeves,  1986). 
These  cells  are  very  specific  in  their 
responses  to  faces,  and  do  not  respond  to 
other  objects,  geometrical  patterns,  or  to 
emotionally  arousing  stimuli,  and  their 
responses  are  relatively  independent  of 
changes  in  size,  orientation,  and  shading  of 
the  facial  features.  There  are  also  other  cells 
in  this  area  that  seem  to  respond  to  specific 
features  on  the  face  such  as  eyes  or  mouth.  It 
has  been  proposed  that  the  “prototypical”  full 
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face  cells  would  act  in  concert  with  the  cells 
that  respond  to  the  smaller  facial  features  to 
form  a  unique  set  of  recognition  units  for 
each  familiar  individual.  Thus,  each  familiar 
face  would  excite  a  particular  neural  template 
when  the  individual  viewed  the  face.  It 
should  be  noted  that  there  are  other  cells  in 
this  area  of  the  brain  that  respond  to  other 
kinds  of  features,  so  it  is  likely  that  the 
processing  in  this  area  is  not  restricted  to 
faces. 

Event-related  Potentials 

ERP  characteristics,  such  as  component 
amplitude  and  latency,  have  been  used  for 
several  decades  as  indices  of  the  amount  and 
kind  of  information  processing  taking  place 
during  the  performance  of  the  experimental 
task.  There  are  two  sets  of  ERP  components 
that  have  traditionally  been  used  in  studies  of 
specific  perceptual  and  cognitive  processes. 
They  are  identified  by  the  polarity  and  the 
latency  of  the  component  relative  to  the  onset 
of  the  stimulus.  First,  there  is  the  P1-N1-P2 
complex.  This  consists  of  a  positive 
component  that  occurs  70-100  msec  after 
stimulus  onset  (PI ),  a  negative  component  at 
100-200  msec  (N 1 ),  and  another  positive 
component  at  200-300  msec  (P2).  These 
components  seem  to  be  related  to  the 
perceptual  salience  of  the  stimulus,  and  the 
amplitude  of  these  components  is  affected  by 
such  factors  as  stimulus  intensity,  and  the 
attention  paid  to  the  stimulus  during  the  task 
(Dinges  &  Tepas,  1976;  Hillyard  &  Picton, 
1979).  The  second  set  of  components  is 
termed  the  late  positive  complex  (LPC).  This 
consists  of  a  variety  of  positive  components 
between  300-500  msec  (P3),  and  a  frequently 
positive  low  frequency  wave,  or  slow  wave  at 
500-800  msec  (S  W).  These  later  components 
have  been  attributed  to  both  the  quality  and 
the  quantity  of  the  cognitive  activity  taking 
place  during  the  task.  The  amplitude  of  these 
components  is  affected  by  such  factors  as 


stimulus  intensity,  probability,  and  meaning, 
as  well  as  the  complexity  of  the  task 
(Duncan- Johnson  &  Donchin,  1982; 
Pritchard,  1981).  In  summary,  there  is  a 
well-known  and  relatively  well-understood 
taxonomy  of  ERP  components  that  may  be 
used  in  the  study  of  the  perceptual  and 
cognitive  processes  taking  place  during  facial 
recognition. 

There  have  been  several  studies  that 
have  examined  the  cortical 
electrophysiological  responses  elicited 
during  object  or  facial  recognition  tasks 
(Davidson,  Schaffer,  &  Saron,  1985; 
Friedman,  Sutton,  Putnam,  Brown,  & 
Erlenmeyer-Kimling,  1988;  Kok  & 
Rooijakkers,  1985;  Lang,  Nelson,  &  Collins, 
1988;  Schulman-Galambos  &  Galambos, 
1978;  Small,  1986;  Stuss,  Picton,  &  Cerri, 
1986).  These  studies  have  generally  found 
that  differences  in  waveform  component 
amplitude  and  latency  parallel  the  differences 
in  response  accuracy  and  latency  in 
behavioral  studies.  That  is,  cortical  ERPs 
waveform  features  seem  to  be  affected  by  the 
same  variables,  such  as  stimulus  familiarity, 
visual  field  position,  and  context,  which 
affect  behavioral  accuracy  and  latency.  These 
findings  have  lent  support  to  the  proposition 
that  cortical  ERPs  directly  index  individual 
performance  during  facial  recognition. 

Methods 

Subjects 

The  subjects  consisted  of  6  male 
employees  of  NPRDC  ranging  in  age  from 
23-49  years  old  (mean  =  35,  SD  =  9. 1 9).  All 
of  the  subjects  had  normal  color  vision  and 
acuity,  or  had  their  acuity  corrected  to 
normal.  Five  of  the  subjects  were 
right-handed,  and  one  subject  had 
mixed-handedness.  None  of  the  subjects 
reported  any  history  of  psychiatric  or 
neurological  disorders. 
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Stimuli 

The  experimental  stimuli  consisted  of  a 
digitized  set  of  both  familiar  and  unfamiliar 
faces,  and  geometric  patterns.  Twelve  faces 
were  created  by  digitizing  same-size 
photographs  in  a  5 1 2  x  480  pixel  format  using 
a  16-level  gray  scale.  The  familiar  faces 
consisted  of  politicians  and  colleagues  in  the 
lab.  The  faces  were  approximately  matched 
for  relative  size,  contrast  (30%),  and  average 
luminance  (10cd/m2).  In  addition  to  the  basic 
set  of  12  faces,  another  set  of  four  faces  was 
produced  from  two  of  the  images  (one 
familiar  and  one  unfamiliar)  by  increasing  or 
decreasing  the  contrast  of  each  of  these  two 
faces.  Thus,  a  total  of  16  different  faces  were 
presented  during  the  experiment.  The 
geometric  stimuli  consisted  of  single  and 
paired  triangles,  and  a  checkerboard  pattern 
with  a  fundamental  spatial  frequency  of 
about  1  cycle  per  degree.  All  of  the  stimuli 
were  viewed  from  a  distance  of  1  meter,  and 
subtended  a  visual  angle  of  about  6  degrees. 
The  subjects  fixated  on  a  small  fixation  point 
located  at  the  center  of  the  screen. 

Recordings 

Electrical  activity  was  recorded  from 
two  occipital  (0 1 , 02),  two  parietal  (P3,  P4), 
two  central  (C3,  C4),  and  two  frontal  (F3,  F4) 
electrode  sites  as  defined  by  the  International 
10/20  system  (Jasper,  1958)  using  an 
Elcctro-Cap  International  El  electrode  cap 
with  tin  electrodes.  These  sites  correspond 
with  the  visual,  sensory-association,  motor, 
and  frontal  areas  of  the  cortex.  All  sites  were 
referenced  to  digitally  linked  mastoids  with 
an  Fz  ground.  Electrode  impedance  was 
reduced  to  less  than  3000  ohms.  EEG  signals 
were  amplified  by  Grass  Model  1 2  amplifiers 
with  a  bandpass  of  0.1-100  Hz  and  a  gain  of 
20,000.  One  second  epochs  (125  msec 
prestimulus  period  and  825  msec 
poststimulus  period)  were  recorded  for  each 


EEG  channel,  digitized  at  128  Hz,  and  stored 
off-line  for  later  analysis.  Eye  blinks  and  eye 
movements  were  detected  during  each  epoch 
by  the  magnitude  of  the  activity  at  sites  F3 
and  F4.  Epochs  in  which  the  amplitude  of  the 
activity  at  these  sites  exceeded  50  microvolts 
were  not  included  in  the  analysis. 

Procedure 

After  electrode  application,  each 
subject  was  instructed  to  sit  comfortably  at  a 
distance  of  1  meter  from  the  display  CRT  and 
to  fixate  on  the  fixation  spot.  Lights  in  the 
recording  room  were  lowered  to  produce  an 
average  background  luminance  of  about  1.0 
cd/m2.  Each  subject  was  allowed  to  adapt  for 
approximately  10  minutes  while  the 
experimental  instructions  were  being  read. 
Each  subject  was  then  presented  with  30 
blocks  of  a  randomized  sequence  of  13 
stimuli  each.  During  each  block,  a  total  of 
eight  faces  and  five  geometric  stimuli  were 
viewed  by  the  subject.  The  stimuli  were 
presented  for  a  duration  of  150  msec,  with  an 
periodic  interstimulus  interval  averaging 
2000  msec.  The  task  of  the  subject  was  to 
report  if  the  face  was  familiar  or  unfamiliar, 
whether  the  single  triangle  was  left  or  right  of 
fixation,  or  whether  the  pair  of  triangles 
pointed  up  or  down  by  pressing  one  of  two 
keys  with  the  right  hand.  After  the  session, 
each  subject  was  asked  to  identify  which 
faces  were  familiar  or  unfamiliar  to  the 
subject. 

Data  Analysis 

For  each  subject,  the  ERP  epochs  were 
averaged  for  each  of  the  16  faces,  and  then 
averaged  across  the  four  familiar  faces.  The 
same  procedure  was  repeated  for  four  faces 
that  were  judged  unfamiliar  for  all  of  the 
subjects.  One  of  the  problems  in  analyzing 
ERP  data  is  the  intersubject  variability  in 
waveform  component  shape.  This  means  that 
it  can  be  difficult  to  determine  the  exact 


11 


IR/IED  FY89  Annual  Report 


IR/IED  FY89  Annual  Report 


location  of  the  peak  of  the  component,  and 
consequently,  the  latency  of  the  component. 
Therefore,  we  chose  to  analyze  the  data  by  a 
method  that  allows  for  this  variability.  For 
every  subject,  the  ERPs  elicited  in  each 
condition  were  divided  into  18  successive  47 
msec  windows  from  46  msec  before  to  800 
msec  after  stimulus  onset.  The  average 
amplitude  of  the  data  points  within  each 
window  was  then  calculated.  Next,  the 
differences  between  the  average  amplitude  of 
each  pair  of  windows  for  the  familiar  and 
unfamiliar  conditions  at  each  recording  site, 
and  the  differences  between  each  pair  of  sites 
for  both  the  familiar  and  unfamiliar 
conditions  were  calculated.  These  difference 
scores  were  used  as  the  dependent  measures 
in  the  analyses.  The  difference  scores  were 
tested  for  significant  deviation  from  zero  with 
a  t-test. 

Results  and  Discussion 

The  grand  average  ERPs  for  the  familiar 
and  unfamiliar  faces  for  one  precentral  site 
(F4)  and  one  postcentral  site  (02)  are  shown 
in  Figure  1 .  The  ERPs  recorded  from  the 
frontal  and  central  sites  had  similar  features, 
and  the  ERPs  recorded  from  the  parietal  and 
occipital  sites  had  similar  features.  However, 
the  ERPs  recorded  at  the  anterior  sites 
(frontal  and  central)  were  somewhat  different 
than  the  ERPs  recorded  at  the  posterioi  sites 
(parietal  and  occipital),  particularly  with 
respect  to  the  very  early  components.  This  is 
most  likely  due  to  the  functional  differences 
of  the  cortical  areas  underlying  each 
recording  site.  The  ERPs  recorded  from  the 
frontal  and  central  sites  for  both  kinds  of 
faces  were  characterized  by  positive 
deflections  at  about  170  (PI)  and  390  (P3) 
msec  after  stimulus  onset,  and  a  negative 
deflection  at  about  280  (N2)  msec.  In 
addition,  the  ERPs  resulting  from  the  familiar 
faces  exhibited  a  late  positive  slow  wave 
(PSW)  with  multiple  peaks  between  600-875 


msec  after  stimulus  onset.  The  ERPs 
recorded  at  the  parietal  and  occipital  sites 
were  characterized  by  positive  deflections  at 
about  105  (PI),  205  (P2),  and  420  (P3)  msec 
after  stimulus  onset,  and  negative  deflections 
at  about  145  (Nl)  and  260  (N2)  msec.  In 
addition,  there  was  a  PSW  for  the  familiar 
faces  at  the  parietal  sites  with  multiple  peaks 
between  500-875  msec,  and  a  late  negative 
slow  wave  (NSW)  at  the  occipital  sites  for  the 
unfamiliar  faces  with  multiple  peaks  between 
600-875  msec.  The  latencies  of  these 
components  at  each  site  and  for  each 
condition  are  shown  in  Table  1.  As  has  been 
seen  in  previous  studies  (Lewis  &  Froning, 
1981),  there  was  considerable  individual 
variation  in  the  waveform  shape  and 
component  latency  of  the  ERPs  recorded  at 
each  of  the  sites,  and  not  all  subjects 
exhibited  all  of  the  components.  In  general, 
we  did  not  analyze  differences  in  component 
latency  due  to  these  large  individual 
variations.  However,  there  was  one  latency 
difference  that  was  prominent  across 
subjects.  At  each  of  the  recording  sites,  the 
N2  latency  was  about  16  msec  shorter  for  the 
unfamiliar  faces  than  for  the  familiar  faces. 

The  first  factor  that  we  examined  was 
the  difference  in  the  windowed  average 
amplitude  of  the  ERPs  elicited  by  familiar 
and  unfamiliar  faces.  There  were  no 
significant  amplitude  differences  between  the 
ERPs  elicited  by  the  two  types  of  faces  during 
the  prestimulus  window  at  any  of  the 
recording  sites.  However,  there  were 
significant  differences  found  at  many  of  the 
poststimulus  windows,  and  these  are  shown 
in  Table  2.  The  number  of  significant 
differences  war,  smallest  at  the  frontal  sites, 
and  this  number  increased  as  the  locus  of 
recording  shifted  toward  the  posterior  sites. 
In  addition,  the  number  of  significant 
amplitude  differences  was  greater  at  sites 
over  the  right  hemisphere  than  at  the 
comparable  sites  over  the  left  hemisphere. 
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Figure  1.  Grand  average  of  ERPs  elicited  by  familiar  (solid  line)  and  unfamiliar 
(broken  line)  faces  at  recording  sites  F4  and  02. 
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Significant  Familiar-unfamiliar  Windowed  Average  Amplitude  Differences 
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There  were  no  significant  differences  for  any 
of  the  17  poststimulus  windows  at  site  F4. 
For  many  of  the  other  sites,  there  were  three 
regions  of  the  ERPs  that  exhibited  significant 
differences.  These  were  at  windows  6-7 
(189-283  msec)  for  all  sites  except  F3  and  F4, 
windows  10-11  (377-470  msec)  for  the 
occipital  sites  only,  and  windows  15-18 
(612-800  msec)  for  all  sites  except  F3,  C3, 
and  P3.  In  all  cases,  the  familiar  faces  elicited 
ERPs  that  were  more  positive  in  amplitude 
than  those  elicited  by  the  unfamiliar  faces. 

The  second  factor  that  we  examined  was 
the  windowed  average  amplitude  difference 
between  the  ERPs  recorded  at  similar  pairs  of 
recording  sites.  This  was  to  determine 
whether  or  not  there  were  any  hemispheric 
differences  in  the  ERPs  elicited  by  each  type 
of  face.  There  were  no  significant 
hemispheric  amplitude  differences  between 
the  prestimulus  windows  in  any  of  the 
conditions.  However,  there  were  significant 
amplitude  differences  it  some  of  the 
poststimulus  windows,  and  these  are  shown 
in  Table  3.  The  number  of  significant 
differences  was  greater  for  the  familiar  faces 
than  for  the  unfamiliar  faces,  and  only  one 
difference  was  found  in  the  parietal  region. 
The  significant  hemispheric  differences  were 
found  to  lie  in  three  regions  of  the  ERPs. 
These  were  at  window  4  (95-142  msec)  for 
the  parietal-familiar  and  central-unfamiliar 
conditions,  windows  7-9  (236-377  msec)  for 
the  frontal  and  central  regions  for  both 
familiar  and  unfamiliar  conditions,  and 
windows  11-18  (424-800  msec)  for  the 
frontal,  central,  and  occipital  regions  for  the 
familiar  faces.  The  ERPs  recorded  over  the 
right  hemisphere  were  more  positive  than  the 
ERPs  recorded  at  comparable  sites  over  the 
left  hemisphere  for  the  frontal,  central,  and 
parietal  regions,  but  this  pattern  was  reversed 
for  the  occipital  region. 


The  results  provide  strong  support  for 
two  conclusions  that  have  been  drawn  in 
previous  behavioral  studies.  First,  familiar 
and  unfamiliar  faces  appear  to  be  processed 
differently  in  the  brain.  The  data  also  suggest 
that  the  processing  networks  are  different, 
rather  than  a  difference  in  the  amount  of 
processing  in  a  single  neural  network.  Across 
sites,  the  most  consistent  difference  between 
the  ERPs  elicted  by  these  two  types  of  faces 
is  in  the  amplitude  of  the  late  slow  wave.  The 
slow  wave  is  more  positive  for  the  familiar 
than  for  the  unfamiliar  faces.  These 
differences  were  greatest  at  the  sites  in  the 
parietal  and  occipital  areas  of  the  right 
hemisphere.  This  is  in  line  with  lesion  studies 
which  have  suggested  that  the  boundary  of 
the  temporal,  parietal,  and  occipital  lobes  is 
the  site  of  processing  during  perceptual  and 
cognitive  recognition.  Second,  there  was  a 
distinct  asymmetry  in  ERP  amplitude  seen 
between  each  of  the  four  pairs  of  recording 
sites.  This  processing  asymmetry  was  most 
prominent  between  the  frontal  sites,  and  least 
prominent  between  the  parietal  sites.  The 
asymmetry  was  greatest  for  the  familiar 
faces,  and  this  suggests  that  there  is 
hemispheric  specialization  of  the  neural 
networks  underlying  facial  recognition.  It 
should  be  noted  that  although  there  were 
large  individual  differences  in  ERP 
components,  many  of  the  significant 
differences  could  be  seen  in  the  ERPs  for  five 
of  the  six  subjects. 

Conclusions 

This  research  has  shown  that  there  are 
measurable  group  differences  in  the 
waveform  components  of  ERPs  elicited  by 
human  faces  based  on  the  familiarity  of  the 
stimulus.  These  differences  are  qualitatively 
and  quantitatively  similar  to  the  findings  of 
previous  behavioral  and  medical  studies.  The 
results  suggest  that  ERPs  may  be  a  useful 
noninvasive  technique  to  assess  the  degree  of 
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learning  on  tasks  which  involve  making 
categorical  distinctions  between  complex 
visual  stimuli.  This  may  allow  direct 
evaluation  of  the  effectiveness  of  different 
training  techniques,  or  the  identification 
those  individuals  who  might  benefit  from  a 
particular  technique.  An  important,  but  not 
yet  tapped  use  of  this  technique  may  be  in 
nonverbal  interrogation  for  personnel 
security  and  reliability  assessment. 

Recommendations 

There  is  a  need  to  continue  exploratory 
research  on  this  experimental  paradigm 
within  the  context  of  the  two  potential 
applications  of  this  technique.  There  are  four 
conditions  that  must  be  met  before  this 
technique  could  be  effectively  used  to  make 
personnel  and  training  decisions.  First,  it  is 
important  to  establish  that  these  differences 
can  be  seen  in  the  ERPs  elicited  by  different 
kinds  of  visual  stimuli  using  different  metrics 
of  recognition.  Second,  it  must  be  determined 
that  the  ERP  components  provide  a  measure 
of  the  degree  to  which  the  stimulus  is  known. 
That  is,  some  measure  of  the  features  of  the 
ERP  must  correlate  with  the  degree  of 
familiarity.  This  means  that  the  waveform 
component  characteristics  must  change 
during  the  learning  process.  Third,  it  is 
important  to  determine  if  correct  ERP 
responses  can  be  elicited  from  individuals 
whether  or  not  they  deny  knowledge  of  the 
stimulus.  Fourth,  it  must  be  established  that 
the  ERPs  that  are  obtained  in  this  paradigm 
can  be  reliably  obtained  under  a  variety  of 
intervening  conditions.  Thus,  the  ERP  would 
be  an  unbiased  estimate  of  the  familiarity  of 
the  subject. 
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Optimization  of  Nonlinear  Objective 
Functions  and  Dynamic  Modeling  in 
Personnel  Distribution 

Iosif  Krass 


In  many  problems  concerning  the  rotation  and  reassignment  of  Navy  personnel, 
a  primary  objective  is  the  distribution  balance  of  people  by  skill  and  pay  grade 
across  locations,  types  of  duty,  and  major  commands.  In  the  Enlisted  Personnel 
Allocation  and  Nomination  System  (EPANS),  this  objective  was  approximated 
heuristically.  However,  in  developing  a  dynamic,  long-term  planning  model  for 
seaishore  rotation,  there  is  the  need  for  a  more  rigorous  solution  procedure. 
Mathematically,  the  distribution  balancing  objectives  are  convex  nonlinear 
functions  that  should  be  addressed  in  the  framework  of  a  multiobjective  opiimization 
problem  over  a  bounded  polyhedron.  The  computer  algorithm  was  developed  that 
converges  to  the  nonlinear  optimal  solution  of  the  distribution  balancing  problem 
through  linear  approximation  techniques  that  do  not  strongly  change  the  original 
polyhedron.  The  algorithm  is  able  to  handle  “ hard  requirements,’'  such  as  minimum 
or  maximum  manning  percentages  for  sea  and  shore  duty,  or  minimum  readiness 
levels  for  specific  ships.  This  approach  was  named  the  “Shadow  Method”  because  it 
consisted  of  shadowing  or  weighting  of  undesirable  area  of  a  nonlinear  penalty 
function  which  should  be  minimized.  The  “Shadow  Method"  was  successfully  tested 
on  the  Navy’s  seaishore  rotation  problem.  A  computer  code  was  developed  to  convert 
the  nonlinear  balancing  problem,  using  the  “Shadow  Method,”  to  a  linear 
optimization  problem. 


Background  and  Problem 

In  many  problems  concerning  the 
rotation  and  reassignment  of  Navy  personnel, 
a  primary  objective  is  the  distribution  balance 
of  people  by  skill  and  pay  grade  across 
locations  and  types  of  duty  (i.e..  Continental 
United  States,  overseas,  sea/shore)  and  major 
commands  (i.e.,  Commanders-in-Chief,  U.S. 
Atlantic  and  Pacific  Fleets).  Liang  and  Lee 
(1985)  approximated  this  objective 
heuristically  and  Liang  and  Thompson 
(1987)  implemented  it  in  a  system  of 


computer  programs,  the  Enlisted  Personnel 
Allocation  and  Nomination  System 
(EPANS),  which  reassigns  enlisted 
personnel.  Their  approach  is,  technically 
speaking,  a  network  implementation  of  a 
convex  nonlinear  objective  function 
linearization  This  convex  function  should  be 
minimized  over  a  polyhedron  defined  bv  a 
personnel  distribution  model.  The  same  kind 
of  problem  arose  in  assessing  a  personnel 
geographic  stability  program  by  Blanco, 
Buletza,  Liang,  and  Sorensen  (in 
preparation),  where  the  concept  of  “fair 
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share”  can  be  mathematically  reformulated  as 
a  minimization  of  a  convex  nonlinear 
function  over  a  polyhedron.  This  “fair  share” 
approach  is  rather  common  in  the  Navy. 
Williams  and  Whisman  (in  press)  discuss  the 
need  for  a  “fair  sharing”  algorithm  connected 
with  allocating  Summer  Cruise  slots  on  Navy 
ships  to  the  Naval  Reserve  Officer  Training 
Corps  (NRC'  ',)  schools.  Distribution 
balancing,  which  is  a  version  of  the  “fair 
share”  concept,  is  also  one  of  the  goals  that 
should  be  reached  in  allocating  personnel  to 
transitory  jobs,  such  as  recruiters  and 
instructors  (see  B-K  Dynamics,  Inc.  (1985)). 
The  problem  gets  even  more  complicated 
when  personnel  distribution  is  considered  for 
more  than  one  planning  period.  For  example, 
the  sea/shore  rotation  problem  discussed  in 
Sorensen  and  Jones  (1984)  or  Chames  and 
Cooper  (1984)  is  a  typical  multiperiod 
distribution  problem.  This  kind  of  problem 
required  dynamic  modeling,  which  can  be 
done  through  network  implementation,  like 
in  Chames  and  Cooper  (1984),  or 
analytically,  using  the  von  Neumann 
approach,  which  is  very  well  known  in 
economic  dynamic  modeling  as  discussed  by 
Morishima  (1964).  As  it  was  shown  in 
Blanco,  Krass,  and  Louie  (1988)  and  Krass 
(in  press),  the  von  Neumann  approach  can  be 
successfully  applied  to  model  a  sea/shore 
rotation  problem  with  fixed  tour  lengths  or 
with  a  flexible  tour  or  credit  system.  Because 
of  the  dynamic  nature  cf  sea/shore  rotation,  it 
was  not  feasible  to  use  the  heuristic  algorithm 
as  in  Lee  and  Liang  (1985)  to  minimize  the 
convex  nonlinear  function.  We  needed  to 
develop  a  new  approach--“shadow  method” 
(Krass,  in  press).  This  method  is  a  further 
development  of  the  “interval"  approach  by 
Chames  and  Cooper  (1977)  or  the  piecewise 
approximation  of  a  convex  function 
minimization  as  it  is  done  by  Danzig  (1963) 
or  Kennington  and  Helgason  (1980).  Hannan 
(1985)  further  discussed  these  methods  of 
minimizing  convex  nonlinear  functions. 


Objective 

The  main  objective  of  this  research  was 
to  develop  an  improved  computer  algorithm 
that  converges  to  the  nonlinear  optimal 
solution  of  a  convex  minimization  problem 
which  does  not  strongly  change  the  original 
polyhedron:  a  network  with  a  few  side 
constraints.  This  algorithm  should  be  able  to 
handle  “hard  requirements,”  such  as 
minimum  or  maximum  manning  percentages 
for  sea  and  shore  duty,  or  minimum  readiness 
levels  for  specific  ships.  Also  in  the  research, 
we  will  develop  a  systematic  method  for 
dynamic  modeling  of  multiperiod  assignment 
problems. 

Shadow  Method 

This  section  provides  the  technical  base 
for  a  method  of  solving  the  convex 
programming  problem,  which  we  called  the 
Shadow  Method. 

Let 

tpO) :  Rm  -»  R' 

be  a  convex  separable  function  with 


m 

<p(y)  =  I  <p*(y*)- 

k=l 

(1) 

Let  B  c  Rn  be  a  bounded  polyhedron  and  A  be 
an  m*  k  matrix.  Consider  the  convex 

program: 

min  <p(y) 

(2) 

such  that 

45 

n 

(3) 

x  e  B. 

(4) 

Assume  that  <p*  is  a  strictly  convex 
continuous  function  which  has  first  and 
second  continuous  derivatives.  (These 
requirements  can  be  weakened,  as  shown  in 
Krass,  in  press).  Let  \ak,  bk]  D  {y4 :  y  = 
b'h-  •  •>>■*,.  •  .,yj  =  Ax\  x  e  B }  and  ^  be 
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the  minimum  of  function  (p*  within  the 
segment  [ak,  bk]-,  that  is, 

<p*(4t)  =  min  <p»(4)-  (■*) 

^ta*.**) 

Without  loss  of  generality,  we  assume 
that  =  0  and  ykfy  =  0. 

Let  {8*,}  for  i  -  1  ,...,/*  be  a  partition 
of  [ak,  6*]  (i.e.,  8*,  <  8U  <  .  .  .  <  8*^),  and 
G  {8*,}.  With  every  8b  we  connect  a  pair 
of  “goal”  variables: 

y*  -  sq  +  s£-  8*,  (5) 

We  construct  the  following  objective 
function  by  using  those  8  variables. 

Minimize: 

m 

X  (X  (tp*  (8*,,+/)  -  (8*,i))  Sfc- 
X  (<P*' (&*,,)--  (6) 

subject  to  conditions  (3)  and  (5).  This 
optimization  problem  we  will  call  the  8 
problem. 

Let  y8  =  (y*,...,  yi)  be  a  solution  to 
the  8  problem,  it  can  be  shown,  that  if  the 
chosen  partition  {8*,}  is  rather  fine,  i.e., 

m^x(8t  ,+/  -  &*,,)  <  E,  (7) 

where  £  is  a  small  positive  number,  then 
solution  to  8  problem  will  be  close  to  a 
solution  of  the  original  problem. 

Lemma  1 

Let  4  =  {si+}  andfh  ={s*;}  together 
with  y8  be  the  solution  to  8  problem,  then 

1)  s*T  =  0  if  y%  <  8*. ;  2)  s£f  =  0  if  yk>  8* , . 

The  proof  of  the  lemma  follows  from 
the  assumption  of  strictly  convex  functions 
<p*.  k~\ , . . . ,  m.  Actually,  in  this  case 


coefficients  of  the  linear  objective  function, 
(4k<$w)  -  <P k  (8*., ))  for  8*  £ £*  or 
(<P*  (8*., )  -  <P*  (8*,/.;))  for  8fa  <  q* ,  are 
positive. 

For  a  partition  {8*}  defined  by  a 
sequence  {8fa},  we  will  refer  A(8*)  = 
max*  (8*_(>y  —  8*  ,j  as  a  module  of  8* . 

Theorem  1 

Lethn  =  (8*)  be  a  sequence  of  partition 
modules  which  converge  to  0  and  y8  be  an 
optimal  solution  to  the  problem  given  in  (6), 
(5),  (3)  using  the  partition  S„.  Then  there  is  a 
sequence  n,;  such  that 

(8) 

where  y  is  a  solution  of  the  problem  (1),  (2). 

For  a  proof  see  Krass  (in  press). 

Let  cfa  =  (p\  (8*/+;)  -  ip\  (8*,, ),  if 
8fa  >  £*,  and  c*,  =  (p  *  (8*,lW)  -  9*  (8*,),  if 
8fa  <  ^t>  then  the  objective  function  to 
minimize  in  the  auxiliary  problem  is, 


m 


which  is  a  generalization  of  “goal”  variables 
method  for  minimization  of  an  absolute  value 
of  a  variable.  (9)  is  also  a  version  of  the 
Chames  and  Cooper  (1977)  “interval” 
method  for  convex  programming. 

If  part  of  constraints  in  B  are  bounded 
constraints  fory*;  that  is, 

ek<yk<  dk, 

where  ek,  dk  G  \ak,  bk ]  then  coefficients  c*„ 
c*(  in  (9)  for  the  case  of  5*,  >  d*  and  8*,  >  e* 
respectively,  should  be  the  "big  M” 
coefficients.  They  guarantee  that  a  feasible 
solution,  if  any,  is  within  the  bounded 
constraints  given  in  (10). 
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In  Figure  1  we  interpret  the  “interval  goal” 
programming  approach  (2),  (5),  (6)  to  the 
problem  (1),  (2),  (3)  by  creating  a  series  of 
penalties  (shadows)  which  connect  with 

partition  (5y . 5$}.  Bigger  penalties 

(darker  shadows)  are  located  further  from  the 
minimum  point  %  =  8^.  Starting  at  point  Stf, 
the  penalty  is  equal  to  “big  M,”  for  x  >  86. 

Model  Formulation 

In  the  von  Neumann  modeling  of  a 
dynamic  system,  time  is  assumed  to  be 
discrete  t  =  1  . . . ,  T,  where  T  is  the  planning 
horizon  for  the  optimization  part  of  the 
model.  The  state  of  the  model  at  the  time  t  is 
a  vector  Jt(/)  in  the  positive  orthant  of 
n  -  dimensional  Euclidean  space:  .  The 

trajectory  in  a  closed  von  Neumann  model  is 
a  sequence  of  state  vectors  x(l), . . . ,  x(T) 
such  that  (x(t),  *(r+l))  e  Z;  /  =  1, . . . ,  T-l, 


where  Z  is  the  set  of  all  feasible  pair 
processes  (technological  set).  The  set  Z, 
which  is  a  subset  of  /f?  *  /?;  ,  is  a  cone 
envelope  of  elementary  processes;  i.e., 

Z={(x,y)=  £  (ai,bi)‘Ui\ui> 0}  (11) 

i=l 

If  C*,y)  €  Z,  then  the  input  vector  x  can 
be  presented  as: 

*  =  £  a‘‘u>>  (12) 
i=7 

and  the  output  vector  y  can  be  presented  as: 
m 

y  =  X  d3) 

i=; 


7"i  ^3  £-1  ^'5  ^6 

Figure  1.  “Shadowing”  a  penalty  function. 
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The  number  u,  is  called  the  intensity  of  the 
i-th  elementary  process. 

To  describe  an  open  von  Neumann 
model  together  with  the  set  of  elementary 
processes  (a„  b,),  i  =  1 , . . . ,  m  we  should  have 

a  set  of  vectors  fit),  i=  1 . T,  which  define 

interaction  between  the  “outside”  world  and 
the  model  with  technological  set  Z.  A 
trajectory  in  the  open  model  is  a  sequence  of 
state  vectors  x(l), . . . ,  x(T)  such  that 

WO,  >'(/))  G  Z;  (14) 

x(t+\)=y(t)  +  /(f)>0.  (15) 

These  relations  mean  that  to  get  from  the 
input  state  x(t)  to  the  output  state  x(r  +  1)  we 
should  get,  first  of  all,  to  a  transition  state  y(f) 
which  should  be  corrected  with  help  of /(f). 
This  transition  state  should  be  such,  that  even 
after  correction  the  vector  x(t  +  1)  will  have 
all  nonnegative  components.  For  example,  if 
/(f)  is  vector  for  uncontrollable  attrition  and 
therefore  /(f)  <  0;  i  =  1 , . . . ,  m,  then  a  feasible 
transition  vector  y(t)  should  satisfy  to  the 
inequalities  >’,(f)  >  /(f);  /  =  1, .  . . ,  m. 

The  state  vector  x{t)  should  describe  all 
particular  cells  in  the  personnel  distribution 
system.  Undereach  cell,  we  model  a  group  of 
people  with  the  same  parameters  of  interest. 
For  example,  if  we  were  interested  in 
studying  a  sea/shore  rotation  problem  with 
fixed  tour  lengths,  then  a  state  vector  „t(f)  = 
(Xipq(t)),  where  /  =  1 ,  .  . . ,  L  is  a  location 
number  and  L  is  total  number  of  locations; 
p  =  1 , . . . ,  P  is  a  pay  grade  number  and  P  is 
number  of  different  pay  grades;  q  =  1 , . . . ,  Q 
is  number  of  planning  periods  which  a  person 
with  pay  grade  p  is  continuously  staying  in 
the  location  /,  and  Q  is  maximal  number  of 
periods  which  a  person  is  allowed  to  stay  at 
the  same  location  without  changing  his/her 
pay  grade. 


Thus  x/p<?(f)  is  number  of  people  which 
are  staying  in  location  / ,  having  pay  grade  p 
and  they  are  on  this  location  already  q 
planning  periods.  Here  under  location  /  we 
mean  not  only  geographic  location  of  a  duty 
but  also  type  of  duty  like  Sea  duty  or  Shore 
duty.  For  example,  in  the  study  of  a  fixed  tour 
sea/shore  rotation  problem  done  by  Blanco, 
Krass,  and  Louie  (1988),  there  were  15 
locations,  (/,  =  15)  but  in  every  cell  pay 
grade  was  fixed  (i.  e.,  P  =  1). 

In  the  case  of  sea/shore  rotation 
problem  with  a  flexible  tour  or  credit  system 
the  parameter  q  is  the  number  of  earned 
credits  and  Q  is  the  maximum  number  of 
credits  allowable  in  the  model.  In  the  study  of 
the  credit  system  by  Krass  (in  press)  in  every 
cell  pay  grade  was  also  fixed  but  there  were 
only  four  locations  ( L  =  4).  In  this  study, 
locations  coincided  with  the  composites 
below: 

1.  CONUS  (Continental  United 
States)  Shore:  /  =  1; 

2.  CONUS  Sea:  /  =  2; 

3.  OUTUS  (Out  of  Continental 
United  States)  Shore:  /  =  3; 

4.  OUTUS  Sea:  /  =  4  . 

We  will  describe  elementary  processes 
in  the  dynamic  model  assuming,  like  we  did 
in  both  the  above  studies,  that  all  persons  in 
the  model  have  the  same  pay  grades.  In  the 
other  words,  processes  of  promotions  or 
demotions  are  external  to  the  model.  In  this 
case  a  state  vector  x! 0  =  (x/lf(t)  €  R}  *Q.  To 
describe  elementary  processes  in  the  model, 
we  will  introduce  a  unit  vector  elq  €  RL*Q. 
We  will  begin  wiih  two  basic  elementary 
processes: 

1.  ( elq ,  e/(?+1)  -  a  person  is  staying  one 
more  period  of  time  at  the  same  location  (in 
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the  case  of  credit  system  this  person  will  earn 
more  credits); 

2.  ( etq ,  el]{)  -  a  person  is  rotating  from 
location  /  to  location  /,,  beginning  a  duty  on 
this  new  location  (losing  all  earned  credits  in 
the  case  of  credit  system).  Here  we  assume 
that  /  */,. 

To  avoid  possible  infeasible  situation, 
we  will  increase  the  dimension  of  the  model 
state  by  one,  adding  one  more  component 
(s  =  L  *  Q  +  1 )  to  the  model:  stock  outside  the 
model  with  unconstrained  capacity.  With  the 
help  of  this  external  stock  we  add  one  more 
elementary  process:  (etq,  es) ,  where  t,  a  unit 
vector  in  the  new  dimension.  This  process 
describes  the  situation  where  a  person  from 
location  /  being  already  q  periods  on  this 
location  (or  with  q  earned  credits  in  the  case 
of  credit  system)  is  leaving  the  Navy.  The 
model  state  x(t )  now  is  in  R^^1. 

In  the  case  of  a  fixed  tour  sea/shore 
rotation  system,  we  need  an  open  model  (i.  e. 
for  any  time  period  t  it  can  be  uncontrollable 
influx  of  people  to  a  location  /  from  “outside 
world,"  or  uncontrollable  outflux  from  this 
location).  In  other  words,  there  is  a  sequence 
of  nonzero  vectors  f(t)  E  RL  * Q,  where  f(t)  = 
(fiq);  and,  if  flq  >  0  it  means  that  we  have 
influx  of  people  to  a  location  /.  In  the  case  of 
influx,  we  assume  that  the  inequality  flq  >  0 
can  hold  only  for  q  -  1  (i.  e.  new  people  can 
only  begin  their  service  on  location  /)■  On  the 
other  hand,  if  flq  <  0  it  means  that  we  have 
outflux  of  people  from  a  location  /.  In  the 
case  of  an  outflux,  we  assume  that  the 
inequality  flq  <  0  can  hold  only  for  q  -  Qi , 
where  Q  is  a  regular  tour  time  for  location  /. 

In  the  case  of  credit  system,  the 
sea/shore  rotation  model  consists  of  two 
parts:  statistical  and  optimization.  The 
statistical  simulation  part  (described  in 
Loffredo,  Helt,  and  Chen  (1988))  models  all 
personnel  flows  external  to  the  optimization 


von  Neumann  model:  all  gains  (accessions, 
promotions-in)  and  losses  (demotions, 
promotions--out,  non-retentions, 
retirements)  to  each  skill/grade/composite 
combination.  An  output  of  this  statistical 
simulation  model  is  an  input  to  the  closed  von 
Neumann  model  which  runs  two  or  three 
periods  to  make  a  new  input  to  the  statistical 
simulation  part. 

We  will  give  more  details  for  the 
optimization  (von  Neumann)  pan  of  the 
credit  system  because  in  this  part  we  are 
using  shadow  method  for  nonlinear 
optimization.  Here  are  some  specifics  of  the 
optimization  model: 

1.  The  time  interval  in  the  model  (the 
planning  period)  is  1  month.  If  a  person 
remains  1  more  month  in  the  same  composite, 
he/she  will  accumulate  additional  credit(s)  to 
his/her  sum  of  credits  (the  first  elementary 
process  in  our  model).  Initial  distribution  of 
credits  is  given  for  each  person. 

2.  There  are  no  recommended  tour 
lengths  per  composite  but  there  are  two  credit 
thresholds,  K,  and  K2.  If  a  person  is  at 
CONUS  Shore  ( /  =  1  )  and  accumulated 
fewer  than  K|  credits,  then  his/her  rotation  is 
undesirable  (the  fewer  credits  earned,  the 
lesser  desirability  to  rotate  a  person).  If  the 
number  of  accumulated  credits  is  more  than 
K,,  then  this  person  is  eligible  for  rotation 
(the  more  credits  exceed  Kt,  the  greater 
desirability  to  rotate  this  person). 

3.  Threshold  K  is  defined  in  an 
analogous  way  for  CONUS  Sea  (1  =  2).  The 
difference  with  this  and  the  previous  case  is 
that  rotation  to  OUTUS  composites,  /  =  3  or 
/  =  4,  is  not  desirable  in  the  current  model. 

4.  All  people  who  started  at  t  =  1  in 
composite  3  or  4  and  are  considered  to  be 
eligible  to  rotate  and  should  be  rotated  at  the 
end  of  the  month.  If  a  person  accumulates 
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more  than  K-.  credits,  then  it  is  more  desirable 
to  rotate  him/her  to  CONUS  Shore  (more 
credits  implies  more  desirability);  in  the  other 
case  it  is  more  desirable  to  rotate  him/her  to 
other  composites  (fewer  credits  implies  more 
desirability). 

This  phenomenon  takes  place  in  the 
optimization  model  because  the  statistical 
pan  of  sea/shore  rotation  model  should  define 
number  of  people  who  should  be  rotated  from 
composite  3  and  4  (those  completing 
overseas  tours).  The  remaining  personnel  in 
composite  3  and  4  are  not  subject  io 
optimization  and  are  only  used  for 
calculating  manning  levels. 


6.  For  every  location  /,  we  denote  P(l) 
and  p(l )  as  the  maximum  and  the  minimum 
manning  percentage.  Thus,  one  of  the  model 
objectives  is  to  satisfy  the  following 
inequality: 

X,(t)  +  CN(l) 

P(0<  - - </>(/), /=U, 3,4.  (18) 

If  p(/)  =  0  or  P([)  =  0  for  some  / ,  then 
the  corresponding  constraint  is  not  active. 
Consuaint  (18)  can  also  be  incorporated  in 
the  shape  of  a  penalty  function  <p(y)  of  ( 1 7)  as 
explained  in  the  previous  section. 

Optimization  Setting 


5.  One  of  the  objectives  is  to  obtain  the 
best  manning  percentage  balance  for  all 
composites.  To  formulate  this  goal  we  will 
introduce  billets  authorized,  BA(l) ,  and 
non-rotating  personnel,  CN{[) ,  for  every 
composite  ( /  =  1,2, 3, 4  ).  If  X,(r)  is  the 
number  of  rotatable  people  in  composite  /  at 
time  r,  then  the  objective  to  obtain  the  best 
manning  percentage  balance  can  be 
formulated  as: 

Minimize 

v  X„(0  +  CN(/,) 

/=/  /,./2  =  l,....4  1  BA(l\) 

X,2(l)  +  CN(l2) 

BMjj 


As  we  already  mentioned,  the  time 
horizon  T  in  the  optimization  model  is  small, 
T  =  2  or  7=3  .  After  every  7  steps  we  use 
the  optimization  model  output  as  an  input  for 
a  statistical  simulation  model  which  defines 
the  losses  and  gains  to  each  skill/grade/ 
composite.  Moreover,  wc  use  overlapping 
planning  intervals  to  overcome  the 
“boundary”  effect  that  is  common  in 
economic  dynamic  planning  (see  Grinold 
(1983)).  That  is,  we  ran  the  optimization 
model  for  at  least  1  month  more  than  it  is 
necessary  for  statistical  simulation  model 
using  x{T  -  1)  or  x(T  -  2)  as  an  input  to  the 
simulation  model.  Correspondingly,  an 
output  of  statistical  simulation  model  is  an 
input  to  closed  von  Neumann  model.  Its 
trajectory  is  defined  by  a  sequence  of  pairs 


where  T  is  the  time  horizon.  This  problem  is 
approximated  as  a  minimization  problem 
having  objective  function: 

T  4  A 0(0  +  CNU) 

1  I  <P( - TT/uT) - *10°-100>’  <17> 

,=1  i=i  BA'l> 

where  tp(y)  is  the  special  convex  function 
discussed  later  (see  Figure  2). 


(x(t),x(t+  1))  G  Z,  t  =  1 . 7-1,  (19) 

where  Z  C  /?^  I  (3  +  2  is  a  convex  cone  hull 
of  the  three  elementary  processes  described 
above.  (Here  L  =  4,  as  we  already 
mentioned). 

If  we  denote  u/<f(r),  v//)?(r),and  wlq(t)  as 
the  intensity  of  the  first,  second,  and  third 
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1 i i i 

•f  »•  H  m 

Figure  2:  Penalty  function  ty(x). 


elementary  processes,  respectively,  then  due 
to  the  model  definition  we  have  equations 

4 

*/*(*)  =  ulq(t)  +  X  vu  (r)  +  w/?(r), 

r=  1, . . . ,  T;  (20) 

4  G 

*/o(f+D=  Z  Iv/i i,(0;  (21) 

/,=i  <?=o 

Vf>  =  if  <7  >  1. 1  =  1 . 7-1, 

which  define  trajectory  x(r),  r=  1,. . .  ,T  with 
initial  condition 

V1)  =  V  (22) 

where  7/(f  is  the  initial  distribution  of  credits 
for  composite  /.  This  distribution  should  be 
initialized  at  the  beginning  of  all  system  runs 
or  coincide  with  an  output  to  the  simulation 
model  after  an  optimization  model  run. 

Variables  X(r)  in  (16)  to  (18)  can  be 
expressed  as  X,{t)  =  Zj=0*/,(0,  t=  1,...  ,7; 


/  =  1,2,3, 4.  Due  to  the  fifth  requirement 
mentioned  earlier  in  this  section,  we  have 
«/,(l)  =  0  and  v/;1<?(r)  =  0  for  /,  /,  =  3, 4;  that 
is,  at  the  end  of  the  first  month  all  people  from 
OUTUS  composites  should  be  rotated  to 
CONUS  composites. 

From  here  we  can  see  that  the  control  of 
a  trajectory  starting  from  an  initial 
distribution  {//?)  can  be  done  through  the 
control  of  intensity  variables  utq(t),  v//((j(r), 

and  w/)?(0;  /,  l}  =  1,2,3, 4;  <7=1 . Q\ 

t  =  1, . . .  ,T.  To  choose  a  “good”  trajectory, 
we  optimize  the  following  objective  function: 

T-l  4  Q 

III  (c}«  •  ulq(t)  +  (23) 

f=  1  /=  1  q= 0 
4 

X  (d/ 1  n  *  v/7  (t)  +  M  .  w,  (0)  + 

/[=!  q  q 

X,(t)  +  CN([) 

BA([)  *10°-100)) 
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Here,  c\q  =  W,  •  cj9  and 

!q  -  K\  if  /  =  1  and  q  >  K] 
q  -  K2  if  /  =  2  and  q>  K2 
0  otherwise 

Parameter  W'i  is  the  weight  of  the  first 
policy.  The  weight  determines  the  level  of 
commitment  to  meet  thresholds  K ,  and  K2  . 
This  policy  will  be  referred  as  the  “credit 
stability”  policy  because  it  gives  the 
minimum  number  of  credits  a  person  should 
obtain  before  rotation.  That  is,  he/she  should 
not  be  rotated  if  that  number  has  not  been 
reached;  otherwise,  rotate  him/her  as  soon  as 
possible. 

As  in  the  case  of  EPANS  we  have  a 
multiobjective  optimization  model.  In  the 
model  which  is  presented  in  the  next  section, 
the  “credit  stability”  policy  is  the  first  of  four 
objectives.  We  approximate  the 
multiobjective  optimization  with  the  help  of 
system  “nonarchimedian”  weights  Wh 
l  =  1, 2,3,4  which  define  the  hierarchical 
order  between  policies  in  a  single  objective 
optimization  problem  defined  by  equations 
(20)  to  (23). 


his  credits  exceed  CONUS  Sea  threshold, 
he/she  should  not  be  rotated  to  OUTUS 
composites.  On  the  other  hand,  if  a  person  is 
in  OUTUS  composites  and  has  not  reached 
the  threshold,  it  is  desirable  to  rotate  him/her 
to  CONUS  Sea. 

Coefficients  c,3/,  and  weight  W2,  which 
are  associated  with  the  second  policy  in  our 
model,  prohibits  the  use  of  undesirable 
routes;  for  example,  route  /  =  3,  /i  =  4  or 
/  =  4,  /,  =  3 ;  that  is,  rotation  from  OUTUS  to 
OUTUS  are  undesirable. 

The  Value  M  in  (23)  is  a  number  which 
considerably  exceeds  any  other  coefficients 
in  the  model.  It  is  the  M  in  the  “big  M 
method.”  It  presents  a  great  undesirability  for 
a  person  to  leave  the  Navy  in  the  optimization 
interval.  Process  #3,  which  corresponds  to 
the  intensity  wlq ,  is  added  to  the  model  to 
avoid  a  problem  of  infeasibility  of 
trajectories. 

Finally,  cp(.r)  is  the  type  of  convex 
function  described  in  the  previous  section. 
The  function  can  be  expressed  in  the  form: 


Coefficients  q2,  in  (23)  can  be 
expressed  as  c?,1?  =  iV,  •  %  +  *2  ■  nr 
where 


(q-  if  1  =  2,  /,  >  3  and  q>Kx 
C/2/1?  =  <K2  -  q  if  /  >  2,  /,  *  2  and  q  <  K2, 

( o  otherwise 

and 


c,3/ 


i? 


if  route  /,  /j  is  undesirable 
otherwise. 


Coefficient  cjt ,  is  a  continuation  of 
“credit  stability”  policy,  which  states  that  if  a 
person  is  in  CONUS  Sea  composite  /  =  2  and 


<p(x)  =  W3  ■  <plC 0  +  W4  •  <p2(.x).  (24) 

The  function  (p1  (jc)  is  designed  to 
satisfy  constraints  (18),  the  policy  of 
“meeting  boundary  percentage 
requirements.”  The  weight  VV3  reflects 
hierarchical  importance  of  this  policy.  As 
indicated  below,  constraints  (22)  are  not 
“solid”;  they  can  be  broken  if  weight  VV3  is 
not  big  enough.  The  function  (p'Ot)  is  defined 
as: 

{0  ifxe  |p(0  •  100,  WOO] 

(x  -  P(/)  •  100)  ifjt  >  P{1)  IOC) 

(p(/)-l()0-x)ifxG  (0,  MO -1001. 


30 


Clearly,  if  P([)  =  0  (i.  e.  composite  /  has  no 
upper  bound  for  manning  percentage),  then  in 
the  definition  of  the  penalty  function  <p'(jc) 
we  should  put  P(l)  =  +oo. 

The  weight  W4  and  the  function  ($?(. x ) 
are  responsible  for  the  policy  of  balancing 
manning  percentage.  Function  qfiix)  in 
Figure  2  is  a  convex  function,  where  the 
penalty  function  cp(x)  is  drawn  in  a  thick  line. 
Here,  P([)  =  1.05  and  p([)  =  .85.  The  rest  of 
function  <p2(jc) ,  outside  the  region  of  [ p(l )• 
100,  P(l)  ■  1.00],  is  drawn  in  dotted  lines.  The 
function  (jp2(x)  has  the  minimum  at  100 
percent,  and  it  is  shaped  to  force  a  composite 
to  fill  the  manning  percentage  to  at  least  95 
percent  before  filling  composite  fur  her  for 
the  purpose  of  manning  balancing  (The 
corresponding  curve  is  much  steeper  before 
95  percent  than  after). 

Network  Representation  of  the  Model 

The  described  model  can  be  presented 
as  a  network  with  side  constraints.  To 
describe  the  “network”  part  of  the  model  we 
define  a  node  for  every  triple  (/,  q,  (),  where 

/=  1 . L\q=  l,...,Q;r=l . T.Thus, 

a  location  /  at  the  time  t  will  be  characterize 
by  Q  nodes.  Together  with  L  *  Q  *T  nodes 
we  define  a  node  which  we  call  the  general 
stock  to  interpret  the  outside  world. 
Equations  (20)  -  (22)  describe  arc 
connections  between  nodes  plus  possible 
supply  at  some  nodes.  For  example,  equation 
(22)  means  that  every  node  corresponding  to 
a  triple  (/,  q ,  1)  has  a  supply  //(?.  That 
coincides  with  the  meaning  of  the  value  Ilq  for 
the  given  /  as  initial  distribution  of  credits  for 
composite  /.  The  right-hand  side  of  (20)  gives 
arcs  connecting  this  node  with  other  nodes  of 
the  model.  The  term  u,q(  1)  represent  the  arc 
which  connects  the  given  node  (/,  q,  1)  with 
node  (/,  q  +  1 ,2)  if  q  <  Q.  This  agrees  with  the 
definition  of  intensity  w/?(l)  as  intensity  of 
the  process  cf  staying  one  more  period  of 


time  at  the  same  location  and  earning  more 
credits.  The  same  node  (/,  q,  1)  is  connected 
with  nodes  (/,,  q,  1),  where  /  ^  /(  which 
corresponds  to  the  variable  v/)/<?(r)  in(20), 
(21). 

In  Figure  3  we  present  a  sketch  of  part  of 
the  network  model  (case  /  =  1 , 2).  This  model 
is  an  analytical  extension  of  the  Chames  and 
Cooper  (1984)  model.  In  our  model  we  set 
the  time  horizon  T  =  3.  The  state  of  the  model 
at  /  =  2  defines  the  input  to  the  simulation 
model  whose  output  in  turn  provides  an  input 
for  the  next  optimization  run.  The  maximum 
number  of  credits  in  the  implementation  is: 

Q  -  140,  and  we  present  results  of  two  runs 
for  each  of  the  two  models  with  Q  =  1 1 1  and 
Q  =  135.  To  handle  the  nonlinearity  in  the 
objective  function  (23),  we  use  the  “Shadow 
Method”  approach  which  approximates 
problem  (20)  -  (23)  by  a  network  with  side 
constraints  given  in  (5),  where  variables  yk 

are  substituted  by  — *  100  and 

=  100.  As  shown  in  Theorem  1,  the 
number  of  variables  in  (5)  can  be  reduced  in 
half  since  variable  sk  is  not  involved  in  the 
objective  function  (6)  if  5*  >  £*.  Thus,  (5) 
can  be  rewritten  as 

yk  -  X  ^8*  (25) 

However,  if  8*  <  then  (5)  should  be 
rewritten  as: 

>Wk+>8*.  (25) 

To  obtain  a  good  heuristic  solution,  we  first 
use  NETSID  code  described  by  Kennington 
and  Whisman  (1987)  to  get  an  optimal 
solution  of  an  integer-relaxed  network 
problem  with  side  constraints;  then  we  round 
up  this  solution  by  using  another  set  of  codes 
prepared  in  Blanco,  Krass  and  Louie  (1988). 
Average  problem  size  is:  500  nodes  in  the 
network  with  3500  arcs  (network  variables), 
and  150  side  constraints  with  250 
non-network  variables.  The  execution  time 
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for  the  FORTRAN  programs  on  an 
IBM4341/12  is  2  to  3  CPU  minutes. 

Future  Developments 

The  dynamic  approach  described  in  the 
paper  can  be  applied  to  many  Navy 
distribution  and  assignment  problems. 
Particularly,  it  can  and  should  be  applied  to 
the  Navy  Manning  Plan,  where  balancing  and 
“fair  share”  fit  the  nonlinear  concepts 
discussed  in  this  paper.  The  shadow  method 
may  also  prove  useful  in  optimizing  fleet 
personnel  readiness.  As  it  is  shown  in  Liang, 
Krass,  and  Thompson  (in  press),  unit 
personnel  readiness  is  nonlinear  convex 
function. 
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Cognitive  Electrophysiology 

Diane  Williams 


Analysis  of  brain  waves  produced  by  subjects  engaged  in  a  memory  task 
provides  a  real-time  window  into  mental  processing.  This  information  is  a  valuable 
addition  to  the  recall  data  that  is  typically  collected  in  memory  research.  This  project 
investigated  a  memory  effect  called  the  suffix  effect.  Subjects  were  asked  to  memorize 
a  list  of  numbers  while  their  brain  waves  were  collected.  The  results  provided 
valuable  information  about  the  typical  paradigm  used  to  investigate  the  suffix  effect 
and  support  other  findings  that  suggest  an  alternate  explanation  of  this  effect. 
Improvements  in  our  ability  to  understand  cognitive  processing  may  lead  to  more 
efficient  Navy  educational  and  training  procedures. 


Research  Goals 

Various  researchers  have  found  that  the 
brain  waves  produced  by  the  subject  while 
studying  an  item  are  systematically  related  to 
the  subject’s  cognitive  processing  of  that 
item.  These  waves  can  provide  an  index  of 
both  the  subject’s  attention  to  the  item  and  the 
extent  of  the  subject’s  processing  of  items. 

The  Cognitive  Electrophysiology 
Project  used  brain  waves  to  supplement  the 
information  provided  by  the  recall  measures 
traditionally  used  to  investigate  short-term 
memory.  These  data  suggest  some  answers  to 
long-standing  questions  abou'  mental 
processing  in  a  standard  short-term  memory 
paradigwi.  A  better  understanding  of  the 
workings  of  memory  could  lead  to  improved 
predictive  capabilities  of  success  in  school 
and  job  performance  by  providing  a  new 
capability  to  distinguish  between  people 
whose  performance  on  traditional  tests  may 
be  very  similar. 

Background 

In  the  task,  the  subject  is  presented  with 
a  series  of  eight  numbers  and  asked  to  recall 


them  in  order.  Two  effects  have  emerged 
from  this  line  of  research.  These  effects  are 
called  the  modality  and  suffix  effects.  The 
modality  of  the  presentation  has  a  substantial 
effect  on  the  subject’s  memory  performance. 
When  lists  of  numbers  are  presented  visually 
or  auditorily,  performance  is  better  for  the 
auditorily  presented  lists.  This  effect  is 
greatest  for  the  last  list  items  and  extends 
back  into  the  lists.  For  auditory  lists, 
performance  is  nearly  perfect  for  the  last  item 
in  the  lists.  For  the  visually  presented  items, 
performance  is  only  about  60  percent  correct 
for  the  last  item.  The  suffix  effect  refers  to  the 
finding  that  if  the  experimenter  says  anything 
after  the  presentation  of  an  auditory  list-for 
example,  “Recall”-ii  impairs  performance 
when  items  are  presented  auditorily.  If  the  list 
was  presented  visually,  the  extra  word  does 
not  consistently  impair  performance. 

Despite  the  great  quantity  of  research  in 
this  field,  the  modality  and  suffix  effects  are 
not  well-understood.  These  effects  were 
originally  attributed  to  a  hypothetical 
memory  structure,  called  Prccategorical 
Acoustic  Store  (PAS)  by  Crowder  and 
Morton  (1969).  This  memory  was  thought  to 
contain  auditory  sensory  information  that  had 
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not  yet  been  categorized  for  semantic 
content,  PAS  accounted  for  the  modality 
effect  in  that  it  offered  subjects  an  additional 
source  of  information  about  the  last  few  items 
in  an  auditor)'  list.  When  an  additional  word 
was  spoken  after  the  list,  the  suffix  was 
thought  to  overwrite  the  last  few  items  in 
memory,  thereby  decreasing  performance. 

The  PAS  explanation  has  come  under 
increasing  criticism  as  the  suffix  effect  does 
not  appear  to  be  confined  to  the  auditory 
modality  as  was  previously  thought.  The 
suffix  effect  has  been  found  for  tactile 
presentations,  lip-read  speech,  and  pictorial 
presentations.  These  findings  have  in 
common  that  information  is  presented  over 
time  or  subjects  arc  required  to  engage  in 
prolonged  uptake  of  information.  This 
suggests  that  the  failure  to  find  a  suffix  effect 
with  visual  presentation  may  not  result  from 
the  modality  of  presentation,  but  may  simply 
reflect  the  different  time  course  of 
information  presentation.  Although  a  spoken 
and  written  word  contain  equivalent 
information,  the  time  course  of  presentation 
is  quite  different.  A  word  takes  substantially 
longer  to  hear  than  to  read.  It  is  possible  that 
the  processing  which  results  from  this  longer 
auditory  presentation  is  an  important  factor  in 
producing  the  suffix  effect. 

An  unusual  presentation  technique  was 
used  to  test  the  hypothesis  that  the  lime 
course  of  presentation -and  consequendy,  the 
time  course  of  infonnation  uptake-is 
important  in  producing  the  suffix  effect 
(Williams,  1982,  1983,  1989a,  1989b; 
Williams  &  Lewis,  1989).  Words  were 
visually  presented  in  a  manner  that  delayed 
the  uptake  of  information.  The  other 
conditions  used  an  auditory  and  a  normal 
visual  presentation.  LRPs  were  collected  to 
provide  infonnation  about  the  processing  of 
each  stimulus.  Elcctrophysiological  data 
provide  infonnation  that  enables 


discriminations  in  performance  that  cannot 
be  made  using  any  other  currently  available 
source.  This  information  will  be  used  to 
determine  the  nature  of  the  processing 
accorded  stimuli  in  each  modality  and  the 
processing  given  to  the  suffix. 

Approach 

Subjects  were  asked  to  recall  lists  of 
eight  digits  presented  sequentially  in  three 
ways  while  their  event-related  potentials 
(ERPs)  to  these  stimuli  were  recorded.  In  the 
auditory  condition,  the  subjects  heard  the 
digits  presented  using  synthesized  speech.  In 
the  nomial  visual  condition,  the  subject’s  saw 
the  word  form  of  each  number  presented 
(e.g.,  “four").  In  the  spelled  visual  condition, 
the  subjects  saw  the  word  presented  with  only 
one  letter  showing  at  a  time. 

The  subject’s  ERPs  were  recorded  while 
they  were  engaged  in  the  memory  task.  These 
potentials  are  extracted  from  the  subject’s 
recorded  ongoing  brain  electrical  activity  (or 
EEG)  by  averaging  several  of  the  segments  of 
EEG  together.  These  segments,  or  epochs,  are 
chosen  by  being  time-locked  to  the  onset  of 
the  stimulus  presented  to  the  subject.  When 
several  of  these  segments  are  averaged 
together,  a  characteristic  waveform  emerges 
from  the  electrical  activity  produced  by  other 
brain  activity.  This  waveform  represents  the 
brain’s  response  to  the  presentation  of  a 
stimulus  or  class  of  stimuli  within  a  certain 
instructional  context,  For  example,  if  the 
subject  is  instructed  to  view  a  series  of  names 
and  keep  track  of  the  number  of  female 
names,  the  ERPs  produced  by  the  brain’s 
response  to  the  names  will  have  a 
characteristic  shape.  Further,  the  ERPs  for  the 
names  that  are  recalled  will  be  different  from 
the  ERPs  for  the  names  that  were  not  recalled 
(Fabiani,  Karis,  &  Donchin,  19K6). 
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Results  and  Discussion 

The  behavioral  data  are  shown  in 
Figure  1.  A  suffix  effect  was  not  obtained  in 
the  normal  visual  condition  but  was  obtained 
both  in  the  spelled  visual  presentation  and  in 
the  auditory  presentation.  Consequently,  the 
suffix  effect  is  not  a  function  of  the  modality 
of  presentation,  but  is  related  to  the  time 
course  of  presentation.  This  suggests  that  the 
time  course  of  information  uptake  is 
important  in  producing  this  effect. 
Differences  between  the  effect  obtained  in 
the  spelled  condition  and  the  auditory 
condition  suggest  that  this  temporal  factor 
may  be  only  one  of  the  factors  that  goes  into 
producing  the  auditory  suffix  effect. 

The  electrophysiological  data  had  two 
main  results.  First,  the  waveforms  for  the 
trials  that  were  followed  by  a  suffix  were 
different  from  the  waveforms  for  the  trials 
followed  by  a  control  stimulus--before  the 
suffix  or  control  had  been  presented.  These 
data  suggest  that  the  standard  methodology 
of  presenting  a  block  of  trials  together  in  each 
condition  that  will  be  followed  by  a  suffix 
may  alter  the  subject’s  processing.  This  lack 
of  randomization  provides  extra  information 
that  allows  the  subject  to  strategically  alter 
his  or  her  processing  during  the  presentation 
of  the  list  items  before  the  suffix  or  control 
has  been  presented.  This  means  that  the  suffix 
and  control  trials  are  not  equivalent  prior  to 
the  presentation  of  the  suffix  or  control 
stimulus.  Consequently,  blocking  the  trials 
together  may  not  provide  an  accurate 
measure  of  the  suffix  effect. 

The  second  main  result  is  that  the  ERPs 
to  the  suffix  and  the  last  list  item  provide 
further  support  for  the  idea  that  the  timing  of 
the  uptake  of  information  is  crucial  in 
determining  the  suffix  effect.  These  data  are 
shown  in  Figure  2.  When  these  data  are 
compared  for  the  visual  condition,  the 


waveforms  diverge  early  in  processing.  This 
suggests  that  the  subject  is  able  to 
differentially  process  the  suffix  in  this 
condition.  This  is  supported  by  the  failure  to 
find  a  suffix  effect  in  this  condition.  However, 
when  these  waveforms  are  compared  for  the 
spelled  visual  condition  and  the  auditory 
condition,  the  waveforms  are  much  more 
similar.  These  conditions  showed  a  suffix 
effect.  These  data  suggest  that  the  earlier  the 
subject  can  differentiate  the  processing  of  the 
suffix,  the  less  effect  the  suffix  will  have  on 
his  or  her  recall. 

The  other  research  that  has  shown  a 
suffix  effect  for  other  than  auditorily 
presented  words  either  are  presented  over  a 
few  hundred  millisecond  interval  or  requiie 
up  to  a  second  of  processing.  This  suggests 
that  a  key  variable  that  may  link  these  various 
suffix  effects  is  the  idea  of  information 
integration  over  time.  A  word  that  is 
presented  visually  in  the  normal  manner  can 
be  processed  quickly  and  consumes  no 
additional  mental  resources.  A  word  that  is 
cither  presented  auditorily,  or  is  spelled, 
requires  more  time  and  resources  before 
being  excluded  from  further  processing.  This 
extra  time  may  determine  whether  or  not  the 
list  will  show  a  suffix  effect.  The 
concentration  of  this  effect  on  the  last  few 
items  in  the  list  may  merely  reflect  their 
fragile  status  in  memory. 

Conclusions 

The  project  obtained  results  which 
suggest  that  the  slower  time  course  of 
auditory  presentation  affects  cognitive 
processing.  When  the  same  information  is 
presented  visually  using  a  slower  time 
course,  an  effect  that  was  thought  to  not  apply 
to  visually  presented  words  was  obtained. 

This  research  also  provided  data  on  a 
basic  aspect  of  cognition.  Performance  on  the 
type  of  serial  learning  task  used  to  investigate 
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Normal  Visual  Presentation 


Spelled  Visual  Presentation 


Auditory  Presentation 


Figure  1.  Proportion  correct  for  the  normal  visual  presentation  condition,  the  spelled  visual 
presentation  condition,  and  the  auditory  presentation  condition. 
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Spelled  Visual  Presentation 


Auditory  Presentation 


1  ~ 

P+1  1 
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Suffix 

Last  List  Item 


Figure  2.  ERPs  to  the  suffix  compared  with  the  last  list  item  from  the  lists  followed  by  a  suffix. 

These  ERPs  are  measured  from  site  Cz  and  reflect  the  average  of  about  100  potentials. 
Positive  is  graphed  down;  the  ticks  represent  100  msec  and  start  100  msec  before  the 
stimulus. 
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the  suffix  effect  is  highly  correlated  (r  =  .66) 
with  performance  on  a  digit  span  task 
(Martin,  1978).  Digit  span  is  one  of  the 
subtasks  that  comprise  the  Weschler  Adult 
Intelligence  Scale,  and  many  other 
intelligence  and  aptitude  tests. 

This  research  has  also  demonstrated  the 
value  of  obtaining  ERP  measures  when 
investigating  cognitive  processes.  These 
measures  can  offer  a  converging  source  of 
data  for  hypotheses  supported  by  the 
behavioral  data  as  was  the  case  with  this 
project,  or  may  supplement  the  behavioral 
data  by  suggesting  distinctions  that  the  ERPs 
show  that  may  not  appear  in  the  behavioral 
data.  The  data  demonstrate  that  ERPs  can  be 
used  to  determine  whether  situations  are 
cognitively  equivalent  for  subjects  and  to 
monitor  the  time  course  of  information 
processing.  ERPs  may  be  useful  in 
monitoring  operator  performance,  predicting 
success  in  school,  and  predicting  job 
performance.  A  thorough  understanding  of 
this  area  may  provide  long-range  benefits  to 
the  Navy. 
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Using  Diagrams  for  Learning  Procedural 

Tasks 


Jerry  L.  Vogt 


Although  there  has  been  a  considerable  amount  of  recent  interest  in  the  role  of 
illustrations  for  understanding  instructional  material,  little  is  known  about  the 
organization  of  diagrams  or  how  they  interact  with  text  to  aid  learning.  The  goal  of 
this  research  is  to  investigate  how  different  types  of  diagrams  assist  students  in 
understanding  instructional  text  so  that  they  can  better  perform  procedural  tasks. 
This  work  involves  a  series  of  experiments  looking  at  the  effects  of  different  diagrams 
upon  the  learning  and  performance  of  different  tasks.  Before  any  experimental 
studies  were  done,  some  preliminary  data  were  gathered  on  the  types  of  diagrams 
used  in  Navy  training  materials  and  their  relationship  to  the  accompanying 
explanatory  text.  These  survey  results  revealed  several  points,  and  in  particular  we 
noted  that  90  percent  of  the  graphics  in  our  sample  of  Navy  training  materials  were 
line  diagrams  intended  to  show  primarily  the  parts  of  a  device,  and  that  for  about 
one-third  of  the  graphics,  the  picture  did  not  match  the  accompanying  text.  The 
experimental  work  has  started  with  a  series  of  experiments  on  the  effectiveness  of 
different  types  of  diagrams,  in  particular  functional  (indicating  the  flow  of 
information )  versus  representational  diagrams  ( merely  showing  placement  of  parts). 
In  one  experiment,  we  found  that  functional  diagrams  produced  better  performance 
than  representational  diagrams  for  initial  learning  of  concepts  and  facts  about  a  gas 
turbine  engine.  We  are  currently  performing  additional  experiments  on  the  effects  of 
diagram  type  on  learning  and  retention,  and  in  particular  are  examining  the 
characteristics  of  the  material  retained  over  longer  intervals. 


Background 

There  has  been  a  considerable  amount 
of  recent  interest  in  the  role  of  illustrations  for 
understanding  instructional  material  (e.g.. 
Willows  &  Houghton,  1987a,  1987b). 
Beyond  the  general  finding  that  text-relevant 
pictures  enhance  comprehension  of  the 
textual  material  (e.g.,  Winn,  1987;  Levin 
Anglin,  &  Carney,  1987),  little  is  known 
about  the  microstructure  of  diagrams  (Mayer, 
1989)  or  how  they  interact  with  text  to  aid 
learning.  The  goal  of  this  research  is  to 


investigate  how  different  types  of  diagrams 
assist  students  in  understanding  instructional 
text  so  that  they  can  perform  different  types 
of  tasks. 

The  instructional  usefulness  of 
diagrams  can  be  viewed  in  several 
perspectives:  as  adjuncts  that  can  assist  in 
processing  at  an  enhanced  level;  as  helps  for 
relearning  procedural  skills;  and  as  helping 
people  form  mental  models  of  device 
operation.  Any  one  of  these  factors  may  help 
explain  why  a  diagram  alone-compared  to 
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the  text-diagram  condition-produces  good 
performance  in  simple  procedural  assembly 
tasks  (Konoske  &  Ellis,  1987;  Schorr  & 
Glock,  1983).  Diagrams  are  probably  more 
relevant  to  a  more  complex  task,  such  as 
inferring  operational  aspects  or 
trouble-shooting  an  operational  problem, 
where  the  functional  understanding  provided 
by  the  graphical  representation  helps  build 
the  appropriate  mental  model  (Kieras  & 
Bovair,  1984).  Moreover,  a  diagram  showing 
the  functional  operation  of  a  device  should  be 
more  effective  than  a  diagram  showing  parts 
for  teaching  complex  procedural  tasks.  The 
research  proposed  here  will  investigate  how 
task  performance  is  influenced  by  diagram 
type. 

Technical  Objective  and  Problem 

The  technical  objective  of  this  project  is 
to  investigate  the  efficiency  of  diagrams  as 
adjuncts  to  learning  procedural  tasks.  On  the 
basis  of  a  review  on  diagram  use  in  Navy 
training  materials  and  of  the  proposed 
experiments,  guidelines  for  the  use  of 
diagrams  in  learning  procedural  tasks  will  be 
provided. 

The  development  of  graphic  materials 
for  Navy  instructional  purposes  has  been 
very  expensive  and  time-consuming.  For 
instance,  Engineering  Systems  School, 

Great  Lakes  NTC  has  some  50;000  slides  on 
file  and  it  is  estimated  that  the  slides  manually 
developed  in  this  library  cost  from  $100  to 
$300  per  slide.  With  the  introduction  of 
computer-assisted  drafting  and  design  (CAD) 
technologies  into  the  curriculum 
development  environment,  the  use  of  slides, 
transparencies,  and  other  graphic  materials 
has  become  much  more  accessible.  However, 
the  increasing  availability  of  graphics 
production  tools  has  prompted  the  need  for 
guidelines  on  when  and  how  graphic 
representations  should  be  used.  This  research 
concerns  itself  with  this  problem--in 


particular,  how  and  when  diagrams  should  be 
used  for  learning  procedural  tasks. 

Procedural  tasks  have  been  chosen 
because  they  represent  a  very  large 
proportion  of  the  sorts  of  skills  required  of 
technical  ratings  in  the  Navy  (Wetzel, 

Van  Kekerix,  &  Wulfeck,  1986).  Moreover, 
there  is  good  evidence  that  procedural  tasks 
decline  considerably,  or  are  forgotten 
altogether,  before  they  can  be  used  on  the  job 
(Hurlock  &  Montague,  1982),  and  the  use  of 
graphic  representations  may  assist  in  the 
long-term  retention  of  many  procedural 
tasks. 

General  Approach 

This  research  involves  a  series  of 
experiments  looking  at  the  effects  of  different 
diagrams  upon  the  learning  and  performance 
of  different  tasks.  Before  any  studies  were 
done,  some  preliminary  data  were  gathered 
on  the  types  of  diagrams  used  in  Navy 
training  materials  and  their  relationship  to  the 
accompanying  explanatory  text.  The 
purposes  of  this  survey  work  were  to  (1) 
choose  an  appropriate  domain  area;  (?.) 
devise  a  classification  scheme  for  diagrams, 
probably  a  representational  vs.  functional  set 
of  categories  based  on  the  work  of  Levin 
(Levin,  Anglin,  &  Carney,  1987);  (3) 
determine  the  prevalence  of  the  types  of 
diagrams  (some  preliminary  observations 
indicate  that  representational  diagrams  are 
about  twice  as  numerous  as  functional 
diagrams);  and  (4)  investigate  the  congruence 
of  those  graphic  representations  with  the 
textual  material. 

The  experimental  work  is  planned  for 
two  directions.  One  series  of  studies  has 
involved  rewriting,  in  a  principled  manner, 
the  diagrams  of  a  selection  from  a  rate 
training  manual,  and  then  assessing  their 
effectiveness  in  aiding  recall  of  fact  and 
concept  learning.  In  the  other  series  of 


45 


IR/IED  FY89  Annual  Report 


IR/IED  FY89  Annual  Report 


experiments,  we  are  examining  performance 
for  either  simple  procedures  (such  as 
assembly  of  a  model)  or  more  complex 
procedures  (such  as  troubleshooting)  when 
given  either  representational  or 
organizational  diagrams. 

Progress 

The  survey  data  has  been  gathered  on 
the  types  of  diagrams  used  in  Navy  rate 
training  materials  and  their  relationship  to  the 
accompanying  explanatory  text.  In  this  work, 
which  is  described  in  detail  in  Vogt.  Gross, 
and  Winnifeld  (1989),  we  surveyed  the  use  of 
graphics  material  in  Navy  training  manuals 
by  systematically  analyzing  material  for  three 
engineering  ratings.  Both  beginning  (3  &  2) 
and  intermediate  ( i  &  C)  rate  training 
manuals  for  three  engineering  disciplines 
(Engineman,  Machine  Repairman,  and  Boiler 
Technician)  were  studied.  Evaluation  factors 
included  the  type,  function,  size,  and 
placement  of  each  graphic  representation;  we 
also  examined  the  agreement  between  the 
graphic  object  and  its  accompanying  text. 
The  results  indicated  that  90  percent  of  the 
graphics  were  line  diagrams  intended  to  show 
the  pans  of  a  device,  and  that  for  at  least 
20-30  percent  of  the  graphic  presentation,  the 
text  and  the  picture  did  not  match.  We  discuss 
these  results  in  terms  of  improved  use  of 
graphics  in  training  and  technical  manuals. 

A  series  of  experiments  on  the 
effectiveness  of  different  types  of  diagrams 
has  been  started,  and  this  work  is  continuing. 
In  one  experiment  that  has  been  completed, 
we  found  that  functional  diagrams  produced 
better  performance  than  representational 
diagrams  for  initial  learning  of  concepts  and 
facts  about  a  gas  turbine  engine.  Our  subjects 
were  Navy  recruits  awaiting  assignment  to 
“A”  school,  and  were  unacquainted  with  the 
gas  turbine  domain.  We  used  a  textual 
passage  accompanied  by  diagrams  with 


either  functional  (i.e.,  flow  of  information) 
labels  or  representational  (i.e.,  parts)  labels. 
While  further  data  analysis  must  be  done,  we 
have  found  that  with  the  functional  diagrams, 
performance  was  enhanced,  both  for  scores 

0 

on  a  multiple  choice  test  immediately  after 
reading  the  material,  and  for  cued  recall 
measures  taken  1  week  after  initial  training. 

Plans 

We  are  currently  performing  additional 
experiments  on  the  effects  of  diagram  type  on 
learning  and  retention,  and,  ir.  particular,  are 
examining  the  characteristics  of  the  material 
retained  over  longer  intervals.  We  are  also 
starting  work  on  performance  tests  where 
subjects  are  performing  assembly  or 
problem-solving  tasks  when  given 
instructional  materials  with  different  sorts  of 
diagrams.  This  work  will  be  complete  by  the 
end  of  FY90. 
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In  addition  to  the  more  than  7000 formal  courses  taught  in  Navy  schools,  there 
is  a  considerable  amount  of  training  conducted  on-the-job  aboard  ship.  Although  the 
Navy  has  courses  and  programs  that  prepare  petty  officers  to  be  leaders  (e.g., 
LMET),  there  is  no  formal  training  on  how  to  be  on-the-job  trainers  (i.e.,  tutors). 
The  goal  of  this  project  is  to  do  the  basic  research  required  to  provide  information 
for  designing  and  developing  a  formal  program  for  teaching  senior  Na\y  petty 
officers  to  be  effective  on-the-job  trainers! tutors.  The  work  is  proceeding  in  four 
phases:  (1 )  analysis  of  on-the-job  training!  tutoring,  (2)  development  of  a  data 
collection  methodology,  (3)  data  collection  on  shipboard  personnel,  and  (4)  analysis 
and  recommendations. 


Background  and  Problem 

In  addition  to  the  more  than  7000  formal 
courses  taught  in  Navy  schools,  there  is  a 
considerable  amount  of  training  conducted 
on-the-job  in  ship  and  shore  based 
commands.  In  peace  time,  the  Navy  is 
heavily  involved  in  training.  This  is 
especially  true  for  new  job  incumbents  and 
for  those  in  jobs  that  change  frequently  or  are 
difficult  to  master  (e.g.,  the  tasks  are 
complex,  there  are  infrequent  opportunities 
for  practice,  etc.).  Much  of  this  training 
occurs  informally  in  one-on-one  or 
one-on-two  or  three  situations,  with  a  senior 
petty  officer  (e.g.,  E-6,  E-7)  working 
with/teaching  seaman  and  seaman  apprentice 
personnel  on/about  shipboard  tasks.  These 
senior  petty  officers  are  in  effect  tutors  and 
are  responsible  for  bringing  “A”  school  (and 
non  “A”  school)  graduates  from  a  novice 
status  to  a  journeyman.  This  involves 
preparing  them  to  take  and  pass  advancement 
exams,  meet  PQS  and  practical  factor 
requirements,  and  perform  their  jobs. 


Although  the  Navy  has  courses  and  programs 
that  prepare  petty  officers  to  be  leaders  (e.g., 
LMET),  there  is  no  formal  training  on  how  to 
be  on-the-job  trainers  (i.e.,  tutors). 

Objective 

The  objective  of  this  project  is  to  do  the 
basic  research  required  to  provide 
information  for  designing  and  developing  a 
formal  program  for  teaching  senior  Navy 
petty  officers  to  be  effective  on-the-job 
trainers/tutors. 

Approach 

The  project  consists  of  four  phases:  (1) 
analysis  of  tutoiing,  (2)  developing  a  data 
collection  methodology,  (3)  data  collection 
on  shipboard  personnel,  and  (4)  analysis  and 
recommendations. 

Phase  1  involved  an  analysis  of  tutoring 
to  determine  the  factors  involved  in  tutoring 
and  the  characteristics  of  good  tutors.  Several 
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researchers  are  currently  investigating  these 
issues  (Fox  1988a,  b,  c;  Gordon,  1988)  with 
tutors  in  college  subjects.  Phase  1  extended 
this  work  to  technical  training. 

In  Phase  2,  a  data  collection 
methodology  was  developed  for  assessing 
tutorial  skills  and  knowledge.  The 
development  process  involved  field 
observations  and  resulted  in  a  paper  and 
pencil  survey  to  be  administered  to  fleet 
enlisted  supervisory  personnel. 

The  questionnaire  is  divided  into  four 
sections.  The  first  concerns  information 
aboui  personnel  the  respondent  supervises 
(e.g.,  number,  rating,  “A”  school  graduates). 
The  second  section  asks  questions  about  the 
respondent’s  Navy  training  background  and 
his  opinion  of  the  training  his/her  personnel 
have  received.  The  third  section  deals  with 
shipboard  tutoring  issues,  and  the  fourth  asks 
for  demographic  information  on  the 
respondent. 

In  Phase  3,  data  will  be  collected  by 
mailing  the  questionnaire  to  petty  officers 
aboard  ship.  The  questionnaire  will  be 
analyzed  in  Phase  4  and  recommendations 
will  be  made  for  a  formal  training  program  in 
tutoring  for  senior  petty  officers  and  for 
modifications  in  instructor  training  to 
enhance  tutoring  skills. 


Progress 

Phases  1  and  2  have  been  completed. 
Plans 

Phases  3  and  4  will  be  completed  in 

FY90. 
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Stabilization  of  Performance  on  a 
Computer-based  Simulation  of  a  Complex 

Cognitive  Task 

# 

Pat-Anthony  Federico 


The  purpose  of  this  research  is  to  study  the  processes  intrinsic  to  the 
stabilization  of  performance  on  a  complex  cognitive  task  (i.e.,  conducting  an  outer 
air  battle).  Subjects  interact  with  an  animated  computer-based,  graphic  simulation. 
They  allocate,  deploy,  and  manage  tactical  assets  in  a  very  large  number  of 
scenarios  to  defend  carrier-based  taskforces  against  hostile,  missile-launching 
bombers.  Concurrent  and  retrospective  verbal  protocols  were  obtained  from  the 
subjects  regarding  their  battle  management.  Performance  during  each  scenario  was 
automatically  assessed  by  the  computer  system  against  16  multivariate  measures. 
Cognitive  and  statistical  analyses  will  be  conducted  to  study  the  processes  of 
acquiring  skill  and  reaching  stabilization  of  performance  on  this  complicated  mental 
task.  Contributions  to  methodology  and  theory  culminating  from  this  research  will 
result  in  improved  operationally  oriented  performance  assessment. 


Background 

Individuals  vary  in  their  rates  and 
manners  of  skill  acquisition,  especially  in  the 
beginning  of  practice,  and  they  reach 
terminal  performance  plateaus  differentially. 
Early  performance  requires  high  conscious 
control  (i.e.,  it  is  slow,  sequential,  effortless, 
limited,  and  directed),  whereas  late 
performance  tends  to  be  automatic  (i.e.,  it  is 
fast,  parallel,  effortless,  and  less  limited  by 
attentional  focus).  Practice  during  the  early 
stages  results  in  dramatic  changes  in  behavior 
(e.g.,  decreasing  performance  variability, 
minimizing  response  time).  With  practice, 
rate  of  improvement  diminishes  and  becomes 
more  uniform  across  individuals  (i.e., 
performance  stabilizes).  For  some  tasks, 
performance  does  not  seem  to  get  any  better 
or  worse  and  curves  which  reflect  the  -  ate  of 
skill  acquisition  of  individuals  appear  to  be 
parallel  (Ackerman  &  Schneider,  1984; 


Jones,  1984;  Schneider,  1984).  Individual 
variability  among  learners  affects  modes  and 
speed  of  skill  acquisition:  Distinct 
experiences,  cognitive  models,  aptitudes,  and 
motivations  can  influence  early  and  late 
performance  differentially. 

Much  of  the  earlier  research  on  which 
the  above  statements  arc  based  was  done  with 
psychomotor  tasks.  A  lot  less  is  known  about 
complex  tasks,  which  are  primarily  cognitive 
in  nature. 

Problem 

Because  many  factors  affect  the  nature 
and  lime  course  of  acquisition,  beginning 
performance  on  complicated  tasks  is  usually 
not  a  good  estimate  of  terminal  performance. 
Since  usually  and  initially  intricate 
performance  does  not  stabilize,  it  may  reflect 
distinct  facets  of  skill  on  different  attempts  to 
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perform  as  indicated  above.  In  other  words, 
estimates  of  performance  are  likely  to 
measure  different  things  on  different  trials  for 
different  people.  Trying  to  separate 
accurately  better  and  poorer  performing 
people,  or  to  determine  consistently  whether 
a  trainee  has  mastered  a  needed  skill  become 
difficult.  This  potential  lack  of  reliability 
impacts  upon  the  predictive  power  of 
computer-based  simulations  for  assessing 
operationally  oriented  skills.  Therefore,  it 
affects  the  validity  of  computer  simulations 
for  job-sample-performance  testing  in 
functional  contexts. 

Technological  Objective 

The  technological  objective  of  this 
research  is  to  conduct  cognitive  and 
statistical  analyses  as  well  as  theoretical 
modeling  to  study  the  process  of  skill 
acquisition  resulting  in  the  stabilization  of 
performance  on  a  computer-based  simulation 
of  a  complex  cognitive  task. 

General  Approach 

Target  Task 

The  target  task  of  this  research  consists 
of  tactically  allocating,  deploying,  and 
managing  fighter  and  supporting  aircraft  to 
defend  an  aircraft  carrier  and  its  escorting 
ships  against  threatening  Soviet  naval  air 
bombers.  This  task  demands  considerable 
practice  before  it  can  be  executed  with  a 
sufficiently  high  level  of  skill  and  becomes 
automatic.  For  the  purposes  of  this  research, 
this  task  is  considered  as  a  test  of  individual 
differences  in  complex  mental  performance. 
In  the  execution  of  this  task,  the  transition 
from  controlled  to  automatic  performance  is 
important.  This  implies  that  what  is  crucial  is 
not  early  but  late  performance  (i.e.,  how  well 
individuals  do  after  extended  practice).  The 
administration  of  numerous  trials  on  this  task, 
together  with  cognitive  and  statistical 


analyses,  will  make  it  possible  to  note  when 
and  how  stabilization  of  performance  is 
achieved  (i.e.,  when  the  research  subjects  no 
longer  show  any  tendency  to  improve  or 
worsen  with  practice). 

Computer-based  Simulation 

Software  tools  were  developed  for 
constructing  computer-based  animated 
graphic  simulations  of  the  actual  radar 
coverage  of  F-14  and  F/A-18  fighters  and 
E2-C  early  warning  aircraft  as  well  as  fuel 
flow  of  these  planes  together  with  KA-6 
tankers.  These  include  probability  of  kill  for 
Phoenix,  Sparrow,  and  Sidewinder  missiles 
that  the  different  fighters  carry  as  well.  The 
capabilities  to  generate  an  infinite  number  of 
raids  from  Soviet  naval  air  bombers  with 
ASMs  (antiship  missiles)  in  different  warfare 
theaters  and  various  carrier  loadouts  in  terms 
of  numbers  of  each  type  of  fighter  and  missile 
on  board  enable  the  creation  of  an  infinite  set 
or  universe  of  tactical  scenarios.  These  were 
used  to  assess  how  well  individuals  manage 
outer  air  battles  to  defend  carrier-based  naval 
task  forces. 

Subjects 

The  research  subjects,  six  F-14  pilots 
and  radar  intercept  officers  at  NAS  Miramar 
and/or  instructors  and  students  from  the 
Tactical  Action  Officer,  Tactical  Warfare 
Overview,  and/or  Staff  Tactical  Watch 
Officer  Courses  from  the  Fleet  Combat 
Training  Center,  Pacific  were  required  to 
allocate,  deploy,  and  manage  fighter  and 
supporting  aircraft  in  order  to  knock  down 
various  numbers  and  mixes  of  hostile 
bombers  before  they  reach  their  respective 
ASM  launch  points.  Each  computer-based 
scenario  will  be  run  in  compressed  or 
accelerated  time;  each  threat  scenario  will  be 
considered  as  a  performance  test  item. 
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Performance  Criteria 

A  subject’s  tactical  performance  during 
simulated  air  battles  will  be  assessed 
according  to  16  multivariate  criteria.  Some  of 
these  are  as  follows:  the  percentage  of 
incoming  threat  aircraft  that  were  detected  by 
F-14,  F/A-18,  and  E2-C  radar  systems;  the 
percentage  of  bombers  that  fighters  placed  in 
missile  LARs  (launch  acceptability  regions); 
the  percentage  of  hostile  aircraft  shot  down  or 
probable  kills;  the  average  range  from  the 
defended  task  force  at  which  threat  aircraft 
were  knocked  down;  the  percentage  of  hostile 
platforms  knocked  down  before  ASMs  were 
launched,  etc. 

Procedure 

Subjects  were  run  on  the 
computer-based  scenarios  of  these 
symbolically  displayed  air  battles  between 
Soviet  bombers  and  U.S.  carrier-based 
aircraft.  How  well  each  allocates,  deploys, 
and  manages  fighters  and  other  supporting 
aircraft  during  the  simulated  battle  was 
assessed  according  to  the  multivariate 
performance  criteria  mentioned  above.  The 
possible  number  of  incoming  raids  or  specific 
threat  scenarios  form  a  practically  infinite 
universe.  Consequently,  the  set  of  simulated 
tactical  scenarios  will  be  considered  as  an 
operationally  oriented,  domain-referenced, 
job-sample,  performance  test.  With  each 
scenario  as  an  assessment  trial,  subjects  will 
be  administered  200  trials  divided  into  20 
blocks. 

Cognitive  Analysis 

During  the  first  trial  of  every  block, 
verbal  protocols  were  obtained  from  the 
subjects  as  they  conducted  the  simulated  air 
battles.  The  analyses  of  these  concurrent 
verbalizations,  as  well  as  retrospective 
reports,  disclose  the  information  needed  by 
the  subjects  while  they  perform  this  complex 


task.  Comparisons  of  the  thinking-aloud 
protocols  and  retrospective  reports  on  the 
first  trial  of  every  block  reveal  the  variability 
in  cognitive  processing  within  as  well  as 
between  subjects  as  they  acquire  skill  (i.e., 
progress  from  controlled  to  more  automatic 
performance  of  the  task). 

Analysis  of  protocols  obtained  early 
and  late  during  practice  on  the  task  indicate 
how  subjects’  cognitive  processes  and 
structures  change  as  their  performances  tend 
to  stabilize.  These  will  reflect  the  cognitive 
correlates  of  the  acquisition  of  stable  task 
performance.  Together  with  a  thorough 
componential  analysis,  the  information 
obtained  from  the  protocol  analysis  will  be 
used  to  construct  a  model  for  performing  this 
complex  task.  This  model  will  be  used  to 
create  a  theoretical  framework  as  well  as 
build  the  basis  of  an  expert  system  for  a 
computer-based  “intelligent  tactician”  that 
will  monitor,  diagnose,  and  assess  the 
conduct  of  simulated  air  battles  to  defend 
carrier  task  forces. 

Statistical  Analyses 

Combining  statistical  procedures  with 
protocol  analyses  and  conceptual  modeling 
will  provide  an  integrated  account  of  the 
cognition  accompanying  the  acquisition  of 
complex  task  performance.  Together  with 
cognitive  analysis  and  theory,  statistical 
techniques  (e.g.,  a  test  for  the  homogeneity  of 
regression  lines)  can  be  used  to  uncover  the 
mental  processes  and  structures  underlying 
the  acquisition  of  stabilization. 

Potential  Products/Transition 

The  potential  products  of  this  research 
are  contributions  to  a  knowledge  base  and 
much  needed  theoretical  framework.  The 
contributions  to  methodology  and  theory 
culminating  from  this  research  can  be 
naturally  extended  or  transitioned  to  the 
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exploratory  development  of  “intelligent  or 
expen”  computer-based  simulation  systems 
to  measure  complex  cognitive  performance 
in  functional  contexts.  Then,  the  predictive 
power  of  this  type  of  performance  assessment 
can  be  determined.  Likewise,  this  follow-on 
work  itself  can  be  transitioned  to  advanced 
development  of  an  intelligent 
computer-based  simulation  system  to  support 
job-sample  performance  assessment  of 
intricate  cognitive  tasks.  This  advanced 
system  would  allow  the  accessing  of 
developed  methodologies,  theoretical 
orientations,  mental  models,  as  well  as 
generic  software  tools  to  implement 
prescriptive  procedures  to  aid  in  the 
production  of  performance  tests  for  complex 
cognitive  tasks. 

Progress 

Cognitive  and  statistical  performance 
data  have  been  collected.  Verbal  protocols 
have  been  transcribed,  coded,  and  analyzed. 
Multivariate  analyses  and  subsequent 


statistics  have  been  computed  on  the 
performance  data.  Cognitive  and  statistical 
analyses  are  being  interpreted.  A  manuscript 
documenting  this  research  will  be  written. 
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Experienced-based  Career  Development 

Robert  F.  Morrison 


Human  resource  specialists  need  to  be  able  to  design  systematically  patterns  of 
assignment  that  lead  to  the  development  of  effective  performance  in  positions  many 
years  after  the  career  development  process  is  begun.  The  objective  of  this  research 
is  to  identify  the  steps  and  the  time  it  takes  an  individual  to  master  a  single 
assignment  and  use  this  as  a  component  in  a  life-span  model  of  experiential  learning. 
This  effort  provides  an  initial  description  (model)  of  the  factors  that  influence  how 
long  it  takes  individuals  to  develop  expertise  within  a  specific  assignment.  When  the 
research  is  completed,  the  Navy  will  have  an  algorithm  to  add  learning  time  and 
performance  level  factors  to  the  present  methodology  used  to  establish  tour  length. 
At  this  time,  manpower  and  permanent-change-of-station  (PCS)  cost  factors  are  the 
major  factors  considered. 


Background  and  Problems 

One  hidden  assumption  with  the  growth 
of  huge,  formal  education  and  training 
programs  is  that  all  efficient  learning  must 
occur  in  a  structured  (classroom-like) 
systematic  way.  Preliminary  research  on 
managerial  positions  challenges  that 
assumption  and  indicates  that  the  majority  of 
learning  occurs  as  a  result  of  work  experience 
(Brousseau,  1984;  Campbell  (personal 
communication)  4  January  1985;  Hall  & 
Fukami,  1979;  Kanarick  (personal 
communication)  3  April  1984;  Lombardo, 
1982;  Morgan,  Hall,  &  Martier,  1979; 
Vineberg  &  Taylor,  1972).  While  a  model  and 
propositions  covering  an  entire  career  have 
been  proposed  (Morrison  &  Hock,  1986),  the 
detailed  attributes  of  its  components  were  not 
adequately  defined.  This  definition  is 
imperative  to  the  adequate  explication  of  the 
career  development  process. 

Since  the  Navy  moved  from  pursuing  its 
primary  warfare  mission  in  the  mid-seventies 
to  a  peacetime  status,  the  demands  for  its 


personnel,  especially  officers,  to  perform 
effectively  in  a  wide  variety  of  tasks  and 
situations  have  increased  markedly.  A 
program  to  encourage  unrestricted  line 
(URL)  officers  in  the  development  of  a 
secondary  skill  (subspecialty)  foundered,  yet 
culminated  in  the  introduction  of  a  material 
professional  community  in  1986  in  response 
to  Congress.  In  1981,  the  Surface  Warfare 
Commanders  Conference  focused  on  junior 
officers  to  increase  their  technical  skills.  In 
1985,  Congress  imposed  the  requirements 
that  all  officers  must  serve  in  joint 
assignments  in  order  to  be  eligible  for 
promotion  to  flag  (0-7). 

This  plethora  of  demands  has  forced 
policymakers  to  shorten  billet  and  command 
tours  until  they  are  frequently  less  than  18 
months.  Such  policies  have  been  designed 
using  manpower  flow  models  without 
considering  their  effect  on  the  officers 
performance  and  career  development.  The 
fleet’s  personnel  readiness  and  the 
effectiveness  with  which  support  activities 
perform  are  affected  directly  by  the 
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opportunity  that  officers  have  to  develop  the 
capability  to  learn  the  requisite  knowledge 
and  skill  of  each  billet  and  to  develop  them  to 
a  level  of  mastery.  Tour  lengths  that  are  too 
short  do  not  provide  the  opportunity  to 
develop  while  ones  that  are  too  long  make 
inefficient  use  of  the  officer  force  and  may 
lower  the  officers  motivation  to  perform  at  a 
high  level  or  learn  new  tasks/jobs. 

Objective 

The  broad  objective  of  this  research  is  to 
develop  a  generic  model  describing  the 
factors  that  influence  how  long  it  takes  an 
individual  to  develop  an  expert-level  of  skill 
in  performing  work.  The  specific  objective  is 
to  develop,  qualitatively  test,  and  modify  a 
preliminary  model  of  the  learning  that  occurs 
while  the  incumbent  is  in  a  leadership 
position. 

General  Approach 

A  literature  search  was  used  to  identify: 
(1)  the  steps  that  an  individual  goes  through 
in  learning  how  to  perform  a  job  to  the  point 
of  mastery;  (2)  the  parameters  that  contribute 
to  the  level  of  performance  at  entry;  and  (3) 
the  factors  influencing  what  is  learned  and 
how  quickly  it  is  learned.  This  information 
was  used  to  form  an  initial  model  of  the 
experiential  learning  process. 

The  preliminary  model  was  tested 
qualitatively  via  repeated  interviews  with  26 
surface  warfare  department  heads  and  8 
executive  officers.  Using  these  data,  the 
initial  model  was  revised  (see  Figure  1)  and 
the  results  prepared  for  publication  (Morrison 
&  Brantner,  1989).  Research  was  designed  to 


test  a  situationally  specific  model  of 
experiential  learning  on  the  population  of 
surface  warfare  department  heads. 

With  the  support  of  OP- 13, 

SURFLANT,  and  SURFPAC,  the  research 
was  initiated  by  requesting  commanding 
officer  and  department  head  data  from  322 
surface  ships.  These  data  included 
perceptions  of  the  factors  presented  in 
Figure  1  and  of  the  extent  to  which  the 
officers  had  learned  their  jobs  regardless  of 
how  long  they  had  been  in  the  positions. 

Plans 

In  FY90,  the  final  data  will  be  collected 
from  the  ships  and  analyzed  using  the 
relationships  depicted  in  Figure  1 .  The  results 
will  be  reported  to  both  the  surface  warfare 
community  and  professional  societies.  If  the 
results  of  the  situationally  specific  test  with 
surface  warfare  department  heads  are 
promising,  the  work  will  transition  into 
exploratory  development  (6.2)  to  test  its  more 
general  applicability  across  the  Navy. 

Expected  Benefit 

The  Navy  spends  millions  of  dollars 
annually  on  PCS  moves  that  are  based 
primarily  on  manning  and  PCS  cost 
considerations.  By  adding  individual  career 
development  and  performance  factors  to  the 
decision  process,  the  readiness  of  the  fleet 
and  the  effectiveness  with  which  PCS  dollars 
are  used  will  improve.  If  training  proves  to  be 
a  significant  factor  in  the  model,  the  model 
will  provide  a  basis  for  evaluating  training 
programs. 
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Figure  1.  Job  learning  model. 
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Application  of  Machine  Intelligence  to 
Generate  and  Test  Causal  Models  on  Large 

Data  Bases 

Stephen  W.  Sorensen 


Discovery  systems  are  desired  to  search  for  relationships  in  large  Navy 
manpower,  personnel,  and  training  (MPT)  data  bases.  These  discovery  systems  must 
be  intelligent  in  model  formulation,  search,  and  evaluation.  A  practical  test  was 
devised  to  see  if  an  existing  discovery  system  could  produce  meaningful  results  on 
complex  and  messy  Navy  MPT  data.  The  TETRAD  II  program,  that  builds  causal 
models,  was  used  for  the  test.  The  results  indicated  that  discovery  systems  hold 
promise  for  the  analysis  of  Navy  data. 


Background 

Throughout  the  Navy  there  are  many 
large  data  bases,  each  containing  hundreds  of 
data  items.  Users  can  access  the  data  using 
data  base  management  systems,  statistical 
packages  and  prc  ams  written  for  a  specific 
application.  A  ty  ical  data  base  is  NPRDC’s 
TRAINTRACK,  a  cohort  data  base 
containing  training  incidents  of  Navy  enlisted 
personnel.  TRAINTRACK  contains  63  data 
elements  and  over  1,400,000  records  and 
occupies  over  650  megabytes  of  disk  space. 
TRAINTRACK  can  be  used  to  test  policies 
about  school  pipelines,  individual  training 
histories,  and  the  utilization  of  training. 

The  problem  with  applying  traditional 
software  analysis  packages  to  very  large  data 
bases  is  an  implicit  assumption  that  the 
analyst  will  exhaust  all  models  relevant  to  the 
posed  problem.  For  at  least  three  reasons,  this 
is  a  shaky  assumption:  (1)  most 
human-generated  models  involve  a  limited 
subset  of  variables,  (2)  inevitable  human  bias 
as  to  the  relevant  variables  will  constrain  the 
search,  so  in  very  large  data  bases  many 
stable  and  potentially  useful  relationships 


never  get  tested,  and  (3)  the  process  of 
checking  a  model  is  labor-intensive:  the 
model  must  be  formed,  the  data  elements 
must  be  restructured  (i.e.  combined, 
normalized,  scaled,  factorialized,  etc.),  the 
test  made,  and  the  test  results  analyzed.  This 
entire  cycle  must  be  repeated  for  every 
model. 

What  is  needed  are  means  to  generate 
meaningful  models,  use  a  search  mechanism 
constrained  only  by  criteria  of  statistical 
replicability  and  causal  interpretability,  and 
report  only  relationships  that  are  both 
interpre table  and  actionable.  The  software 
must  have  intelligence  in  model  formation, 
search,  and  evaluation.  The  technology 
would  be  useful  not  only  for  Manpower, 
Personnel,  and  Training  (MPT)  data  bases  but 
also  for  any  large  Navy  data  base.  Discovery 
systems,  such  as  those  described  by  Langley, 
Simon,  Bradshaw,  and  Zytkow  (1987)  may 
be  able  to  generate  these  models. 

Objective 

This  project  used  an  existing  discovery 
program  to  automatically  search  for  causal 
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models  in  two  Navy  data  bases.  The 
discovery  method  should  be  effective 
because  it  can  repeatedly  examine 
relationships  while  a  person  is  restricted  to 
only  a  few  attempts.  However,  Navy  data  is 
often  complex  and  messy;  therefore,  a 
practical  test  was  necessary  to  determine  the 
feasibility  of  the  discovery  approach. 

Approach 

The  TETRAD  II  program  ( Glymour, 
Scheines,  Spirtes,  &  Kelly,  1987;  Spirtes, 
Schemes,  &  Glymour,  in  prepartion)  has 
shown  promise  in  discovering  causal 
structure  in  non-experimental  data.  Two  data 
sets  were  developed  for  the  tests.  In  the  first, 
the  TETRAD  II  program  searched  for  causes 
of  attrition  and  setback  in  the  Navy’s  air 
traffic  control  (AC)  school  pipeline.  The  data 
for  the  study  was  taken  from  the 
TRAINTRACK  file  and  the  Survival 
Tracking  File.  NPRDC  arranged  the  data  and 
calculated  the  necessary  covariance  matrices 
for  the  program.  The  program  was  run  at 
Camegie-Mellon  University. 

For  a  second  data  set,  NPRDC  used 
results  of  a  survey  that  collected  opinions  on 
the  recruiters’  quality  of  life  with  particular 
emphasis  on  the  demands  of  their  jobs.  The 
survey’s  ultimate  goal  was  to  construct 
models  to  change  policy  and  improve  the 
effectiveness  of  recruiting.  NPRDC  scaled 
and  arranged  the  data  for  use  in  TETRAD  II. 
The  program  was  run  at  Camegie-Mellon  to 
obtain  causal  models  on  the  recruiters’  job 
satisfaction  and  effectiveness. 

Results 

The  discovery  methods  have  been 
effective  on  the  Navy’s  data.  In  a  simple 
initial  trial,  TETRAD  II  was  able  to  discover 
the  components  from  the  Armed  Service 
Vocational  and  Aptitude  Battery  (ASVAB) 
that  make  up  the  Armed  Forces  Qualifying 


Test  (AFQT)  scores.  The  program  was  given 
seven  ASVAB  scores  for  the  individuals  in 
the  AC  data,  but  only  three  of  the  scores  are 
used  in  .AFQT.  In  spile  of  likely  common 
causes  in  the  test  scores,  TETRAD  II 
correctly  picked  the  three  ASVAB 
components. 

In  a  more  complicated  trial,  TETRAD  II 
was  asked  to  predict  an  outcome  variable 
(non-academic  failure,  academic  failure,  or 
pass)  from  the  following  input  variables: 
AFQT,  number  of  academic  setbacks;  years 
of  education;  ASVAB  score  on  mechanical 
comprehension;  ASVAB  score  on 
mathematical  knowledge,  gender,  length  of 
service,  and  number  of  interruptions  of 
training.  Unfortunately,  the  explanatory 
variables  accounted  for  only  about  10  percent 
of  the  variance  of  the  outcome  variable  and 
the  model  did  not  seem  useful. 

An  alternative  problem  used  the 
outcome  variable  as  a  classification  and  the 
goal  was  to  minimize  misclassification. 
TETRAD  II ’s  model  gave  good  insight  into 
how  misclassifications  occur.  Using  the 
model,  it  was  shown  that  by  restricting 
admission  criteria  and  training  schedule,  the 
Navy  might  reduce  academic  and 
non-academic  failures  from  34.5  percent  to 
20  percent.  Practical  applications  depend  on 
getting  enough  applicants  for  AC  training 
that  meet  the  restrictions. 

Further  tests  are  being  run  with  the  AC 
data  set  and  the  recruiter  data  set. 

Plans 

The  results  from  this  project  were 
encouraging  for  applications  of  discovery 
methods.  The  6.1  project’s  results  will  be 
used  in  the  6.2  project  Training  Management 
Expert  Systems. 
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Loss  Forecasting  With  Empirical  Bayes 

Estimators 

James  P.  Boyle 


Manpower  planners  within  the  Navy  and  Marine  Corps  require  accurate 
personnel  loss  forecasts  to  generate  executable  personnel  plans.  Traditionally,  the 
method  of  least  squares  has  been  a  popular  method  of  estimating  parameters  in  loss 
forecasting  models.  In  recent  years,  a  group  of  statistical  techniques  known  as 
empirical  Bayes  methods  have  emerged  as  competitors  to  the  least  squares 
approach.  This  research  effort  was  designed  to  test  the  hypothesis  that  empirical 
Bayes  estimators  generate  more  accurate  loss  forecasts  than  standard  least  squares 
estimators. 


Background  and  Problem 

Military  manpower  systems  are 
“vacancy  driven.”  New  personnel  are 
recruited  and  current  personnel  are  promoted 
when  vacancies  are  created  by  various  types 
of  personnel  losses.  Plans  to  achieve 
targeted  end-strengths  and  satisfy  budget 
constraints  are  good  only  if  projected  losses 
come  close  to  actual  losses.  Effective  force 
planning  depends  heavily  on  accurate  loss 
forecasts. 

While  there  is  a  wide  variety  of  loss 
forecasting  models  in  use,  the  models  can  be 
grouped  broadly  into  two  categories- 
reflecting  quite  different  viewpoints.  The  first 
group  consists  of  time  series  models  that 
assume  future  losses  and  are  entirely  a 
function  of  past  loss  behavior.  The  second 
group,  comprised  of  econometric  models, 
explain  future  loss  behavior  in  terms  of 
exogenous  variables  (e.g.,  military 
compensation,  civilian  employment 
conditions). 

Although  different  in  spirit,  the  two 
approaches  share  a  common  characteristic. 


Both  contain  parameters  that  must  be 
estimated  before  any  forecasts  can  be 
obtained.  A  popular  method  of  estimating 
these  parameters  has  been  least  squares. 

However,  in  recent  years,  the  popularity 
of  least  squares  has  been  challenged  by  a  set 
of  techniques  known  as  "empirical  Bayes” 
methods.  The  emergence  of  these  “empirical 
Bayes”  methods  is  due  largely  to  a  series  of 
papers  published  by  Bradley  Efron  and  Carl 
Morris  from  1972  to  1977.  These  papers 
contain  both  theoretical  and  empirical  results 
recommending  empirical  Bayes  estimators 
over  least  squares  estimators.  A  book  by  G. 
G.  Judge  and  M.  E  Bock  (1978),  which 
develops  empirical  Bayes  estimators  in  the 
General  Linear  Model,  and  a  recent  article  by 
George  Casella  (1985)  also  confirm  the 
superiority  of  empirical  Bayes  estimators. 

Objective 

The  objective  of  this  research  was  to 
test  the  hypothesis  that  empirical  Bayes 
estimators  of  parameters  in  loss  forecasting 
models  lead  to  better  forecasts  than  those 
based  on  least  squares  estimators. 
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Approach 

The  binomial  model  was  applied  to 
FY81  through  FY88  loss  rate  data  arranged 
by  38  occupational  fields  (OCCF)  and  9  pay 
grades  (PGs).  A  beta  prior  distribution  was 
assumed,  and  the  empirical  Bayes  estimates 
of  loss  rates  were  generated  using  the  method 
of  moments.  Loss  forecasts  based  on  the 
empirical  Bayes  estimates  (transformed 
rates)  were  compared  to  forecasts  based  on 
the  usual  least  squares  estimates 
(untransformed  rates). 

Results 

Four  forecasting  methods  were  applied 
to  each  of  the  38  (OCCFs)  x  9  (PGs)  =  342 
loss  rate  series:  (1)  the  naive  method  that 
forecasts  next  year’s  rate  with  this  year’s  rate, 
(2)  a  simple  average  of  historical  rates,  (3)  a 
weighted  average  method,  and  (4)  an 
exponential  smoothing  method.  Each  method 
was  applied  to  both  the  untransformed  and 
transformed  rates.  A  mean  square  error 
criterion  was  adopted  to  assess  forecast 
accuracy.  Table  1  summarizes  the  results.  For 
example,  when  the  naive  method  was 
applied,  the  transformed  rates  outperformed 
the  untransformed  rates  in  261  of  the  342 
series  or  76.4  percent  of  the  time.  Details  of 
the  FY89  effort  will  appear  in  a  report  being 
drafted  entitled  Shrinkage  Estimates  of 
Marine  Corps  Enlisted  Personnel  Loss  Rates. 

Also  in  FY89,  some  additional  work  on 
properties  of  empirical  Bayes  estimators  was 
completed.  This  work  was  presented  at  the 
annual  meeting  of  the  American  Statistical 


Association  (ASA).  A  paper  describing  this 
research  is  to  appear  in  the  ASA  1989 
proceedings  of  the  Business  and  Economics 
Statistics  section. 

Table  1 


Forecasting  Performance  of  Empirical 
Bayes  Estimates 


Losses 

Wins 

Ties 

Win*  tndTiei 

Model 

% 

% 

% 

% 

Naive 

17.5 

76.4 

6.1 

82.5 

Simple 

Average 

25.8 

64.4 

9.8 

74.2 

Weighted 

Average 

28.1 

64.2 

7.7 

71.9 

Exponential 

Smoothing 

24.7 

68.6 

6.7 

75.3 

Plans 

Results  from  this  project  wiil  transition 
into  an  existing  6.2  project  Marine  Corps 
force  management  forecasting,  and  an 
existing  6.3  project  Marine  Corps  enlisted 
planning  system. 

Expected  Benefit 

Force  management  decisions  impact 
billions  of  dollars  in  manpower 
appropriations.  Since  personnel  loss 
forecasting  is  critical  to  successful  force 
management,  this  effort  commands 
considerable  leverage.  Benefits  are  expected 
to  take  ti  e  form  of  personnel  plans  that 
achieve  s  rill  and  experience  levels  required 
to  sustain  readiness  and  avoid  budget  crises. 
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Military  Recruit  Quality  and  the  Minimum 

Wage 
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0 


The  Congress  is  currently  debating  raising  the  federal  minimum  wage.  Several 
states  have  already  raised  their  minimum  wages.  Proposed  federal  legislation 
includes  a  subminimum  " training  ”  wage  and  a  timetable  for  future  minimum  wage 
increases.  Those  affected  by  the  minimum  wage  and  the  low  end  of  the  wage 
structure  are  the  very  same  individuals  the  military  actively  recruits  (i.e.,  17-21  year 
old  males  and  females).  This  effort  develops  an  economic  model  that  outlines  the 
effects  of  increases  in  the  minimum  wage  on  the  quantity  and  quality  of  military 
recruits. 


Background  and  Problem 

One  of  the  key  factors  influencing  an 
individual’s  decision  to  enlist  in  the  military 
is  his/her  alternative  civilian  wage.  For  many 
of  these  young  people,  their  potential  civilian 
job  will  be  their  first  full-time  work,  and  will 
likely  earn  the  minimum  wage  at  the  outset. 
Congress  is  currently  debating  raising  the 
federal  minimum  wage.  The  last  time  it  was 
increased  was  in  January  1981.  Moreover, 
current  proposals  to  increase  the  minimum 
include  a  so-called  “training  wage”  for  those 
with  less  than  6  months  work  experience. 

The  increase  in  the  minimum  stirs 
debate  over  many  economic  issues.  For 
example,  because  the  minimum  wage 
represents  the  lowest  level  in  any  wage 
structure,  :ncreasing  it  may  mean  increasing 
the  wages  of  other  employees.  Here,  “wage 
distance”  between  the  various  levels  within  a 
wage  structure  may  need  to  be  maintained. 
Other  economists  have  argued  that  wage 
structures  differ  by  level  of  education, 
experience,  occupation,  and  industry. 
Therefore,  changes  in  the  minimum  wage 


may  have  differential  effects  on  certain  labor 
groups. 

The  minimum  wage  does  not  apply  to 
all  economic  sectors.  That  is,  minimum  wage 
coverage  is  not  complete.  For  example,  farm 
workers  are  not  covered.  With  incomplete 
coverage,  economists  have  debated  the  short- 
and  long-run  effects  of  increases  in  the 
minimum.  One  side  of  the  debate  argues  that 
increases  in  the  minimum  wage  drives 
workers  out  of  the  covered  sector  into  the 
uncovered  sector  resulting  in  lower  wages  in 
the  uncovered  sector.  The  other  half  of  the 
debate  believes  that  the  higher  minimum 
wage  attracts  labor  from  the  uncovered  sector 
into  the  covered  sector  resulting  in  higher 
wages  in  the  uncovered  sector. 

So  what  does  this  have  to  do  with  the 
military?  First,  the  military  competes  with 
the  private  sector  for  the  young  people 
seeking  those  entry  level  jobs.  Second,  the 
military  is  one  of  the  largest  uncovered 
sectors.  Thus,  whether  military  wages  rise  or 
fall  is  crucial  to  military  manpower  supply. 
Third,  in  addition  to  the  number  supplied,  an 
important  issue  for  military  training  is  the 


supply  quality.  Under  either  theory,  what  is  regular  minimum  and  subminimum,  can  be 

the  quality  of  the  labor  in  the  uncovered  expected  to  affect  both  the  level  and  quality 

sector?  Does  the  military  change  formal  mix  of  accessions.  It  should  be  noted  that  an 

school  entrance  requirements,  alter  training  economic  model  that  includes  a  subminimum 

curricula,  institute  more  remedial  courses,  wage  set  below  the  regular  minimum  has  yet 

lengthen  course  instruction,  academically  to  be  developed. 


setback  more  students  more  often,  or  change 
weapons  systems  drastically?  Some  of  these 
alternatives  require  changes  in  training 
resources.  Last,  the  decline  in  the  pool  of 
nonprior  service  personnel  has  been 
well-documented.  What  is  the  interaction  of 
this  decline  in  labor  source  and  the 
increase  in  the  minimum  wage? 

Objective 

The  objective  of  this  research  is  to 
develop  an  economic  model  of  military 
supply  quantity  and  quality  and  the  proposed 
changes  in  the  federal  minimum  wage. 

General  Approach 

Because  raising  the  minimum  wage 
impacts  many  economic  research  issues,  it 
will  be  necessary  to  determine  their 
relationship  to  a  military  supply  model. 

Little,  if  any,  research  on  the  minimum  wage 
discusses  its  impact  on  the  military.  A  review 
of  the  literature  in  this  area  will  highlight  the 
salient  points  that  may  relate  to  a  military 
supply  model. 

A  theoretical  model  that  relates  the 
enlistment  supply  of  various  groups  to 
changes  in  the  minimum  wage  structure  will 
be  developed.  Changes  in  the  minimum  wage 
structure,  in  the  form  of  increasing  both  the 


Results 

Because  this  research  will  not  produce 
any  empirical  estimates,  the  model  will 
suggest  the  methodology  for  developing  an 
empirical  model  and  will  provide  qualitative 
estimates  of  the  likely  direction  of  effect.  If 
adequate  estimates  of  military  accession  and 
training  costs  exist,  calculations  of  changes  in 
these  costs  as  a  result  of  changes  in  the 
minimum  and  subminimum  wage  will  be 
generated. 

Expected  Benefit 

The  services  could  react  passively  to 
this  change  by  accepting  whatever  the 
resulting  level  and  mix  of  accessions  happens 
to  be,  or  the  services  could  respond  by 
altering  their  own  policy  instruments  to 
contend  with  changes  induced  by  the  altered 
minimum  wage  structure.  For  example,  the 
services  could  respond  by  varying  pay, 
numbers  of  recruiters,  advertising 
expenditures,  enlistment  bonuses,  or  quotas 
so  as  to  keep  fixed  the  number  of  high  quality 
recruits.  Computing  the  cost  of  doing  so 
requires  knowledge  of  (1)  how  each  of  these 
factors  affects  high  quality  enlistments  and 
(2)  the  cost  of  changing  each  of  these  factors, 
as  well  as  (3)  the  enlistment  effects  of 
changes  in  the  minimum  wage  structure. 
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David  G.  Huntley  is  a  Mathematical  Statistician  in 
the  Manpower  Systems  Department.  He  earned  a  dual  degree 
with  distinction  in  French  literature  and  mathematics,  as  well  as 
an  M.S.  in  Operations  Research,  from  Stanford  University.  He  is 
a  member  of  Phi  Beta  Kappa.  His  research  interests  include 
applications  of  neural  nets  to  forecasting,  stochastic  processes 
and  optimization.  His  most  recent  work  will  be  published  in  the 
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Applications  of  Neural  Net  Technology  to  the 
Forecasting  of  Time  Series 

David  G.  Huntley 


The  purpose  of  this  research  is  to  discover  whether  a  neural  net  approach 
provides  more  accurate  forecasts  than  conventional  techniques  such  as  Box-Jenkins 
and  ARIMA.  The  “ turning  point"  problem  in  forecasting  is  defined  as  the  inability  of 
conventional  forecasting  methods  to  anticipate  fluctuations  in  the  data;  they  are 
excellent  at  predicting  trends,  but  when  the  trends  change,  there  is  a  delay  in 
recognizing  the  change.  The  goal  of  the  research  is  to  examine  whether  neural  nets 
are  more  sensitive  to  fluctuations  in  the  data. 


Background  and  Problem 

The  Navy’s  ability  to  meet  its 
manpower  needs  is  restricted  by  the  size  of 
the  personnel  budget.  To  keep  personnel 
expenditures  within  budget,  it’s  necessary  to 
make  accurate  forecasts  of  the  personnel 
inventory  by  grade  and  length  of  service 
(LOS).  Forecasting  methods  need  to  be  very 
accurate  because  small  errors  in  a  forecast  of 
the  Navy’s  18  billion  dollar  military 
personnel  budget  can  result  in  very  large  cost 
overruns. 

One  approach  to  this  problem  was 
developed  in  the  1970s.  The  Naval  Personnel 
Pay  Predictor,  Enlisted  (NAPPE)  is  a  time 
series  model  of  the  enlisted  population 
arrayed  by  LOS  in  years.  Transition  rates 
from  LOS  cell  j  to  LOS  cell  j  +  1  are  forecast 
by  the  NAPPE  m  del,  using  Box-Jcnkins 
techniques.  NAPPE’s  predictions  for  mean 
LOS  of  the  work  force  indicated  a  high 
degree  of  accuracy,  and  this  is  the  measure 
we  for  cast  by  the  neural  net  approach 
discussed  in  the  next  section.  The  accuracy  of 
the  forecasts,  in  any  case,  tends  to  diminish  as 
the  lead  time  increases. 


Neural  Net  Forecast 

The  neural  net  model  used  was  part  of  a 
softw'are  package  by  NeuralWorks.  The 
forecasts  were  run  on  an  IBM  PC  XT.  Several 
network  types  were  tried.  These  models  use 
one  or  two  “hidden  layers”  to  generate  an 
internal  representation  of  the  pattern  to  be 
matched.  Such  a  configuration  allows  an 
arbitrarily  difficult  pattern  association:  any 
set  of  unambiguous  input/output  pairs  can  be 
learned. 

The  learning  rule  is  basically  an 
error-correction  algorithm.  The 
back  propagation  paradigm  used  in  the 
research  uses  a  gradient  descent  method  to 
find  (hopefully)  a  global  maximum  in  its 
search  to  optimize  the  match  between  input 
and  output.  One  of  the  problems  of  a  gradient 
descent  algorithm  in  neural  networks  is  that 
of  setting  the  learning  parameter.  If  the 
parameter  is  set  too  high,  the  network  can 
oscillate--not  converging  at  all.  On  the  other 
hand,  a  small  learning  rate  may  make  the 
network  take  a  very  long  time  to  converge. 
Part  of  the  problem  was  to  discover  an 
appropriate  rate. 
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Due  to  limitations  in  the  size  and  speed 
of  the  computer,  the  back-propagation 
algorithm  used  took  an  extremely  long  time 
to  converge.  After  months  of  training,  a 
suitable  differentiation  in  cell  size  showed  the 
process  beginning  to  converge.  Rapid 
improvisation  and  turnaround  were 
impossible  to  achieve,  so  no  definitive 
conclusions  were  reached. 


Huntley,  D.  G.  (10  April  1990).  Neural  nets: 
An  approach  to  the  forecasting  of  time 
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See  Appendix  B,  p.  B-l. 
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Transitions 


Independent  Research 

Experienced-based  career  development  (0601152N.R000.06)  will  transition  into  exploratory 
development  6,2  research. 

Application  of  machine  intelligence  to  generate  and  test  causal  models  on  large  data  bases 
(06011 52N.R0001  07)  will  be  used  in  the  6.2  project  Training  management  expert  systems  and 
in  the  exploratory  development  6.2  project  Developing  measures  of  effectiveness  for  knowledge 
discovery  systems  (0602936N.RV36I27.08). 


Independent  Exploratory  Development 

Forecasting  with  empirical  Bayes  estimators  (0602936N.RV36I27.05)  will  transition  into  an 
existing  6.2  project.  Marine  Corps  force  management  forecasting  and  an  existing  6.3  project. 
Marine  Corps  enlisted  planning  system. 
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Presentations 


Independent  Research 

Williams,  D.  (1989b).  ERPs  and  the  suffix  effect.  Poster  presented  at  the  annual  meeting  of  the 
Psychonomic  Society,  Atlanta,  GA. 

Williams,  D.  &  Lewis,  G.  (1989).  Electrophysiology  of  human  memory.  Paper  presented  at  the  2nd 
Navy  Independent  Research! Independent  Exploratory  Development  Symposium ,  White  Oak, 
MD. 


Independent  Exploratory  Development 

Huntley,  D.  G.  (April  1990).  Neural  nets:  An  approach  to  the  forecasting  of  time  series.  Paper 
presented  at  the  Conference  on  Advanced  Computing  for  the  Social  Sciences,  Williamsburg, 
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Publications 


Independent  Research 

Callahan,  J.,  &  Sorensen,  S.  (1989).  Rule  induction  for  group  decisions  with  statistical  data,  an 
example.  Submitted  for  publication. 

Federico,  P-A,  (in  press).  Student  cognitive  attributes  and  performance  in  a  computer-managed 
instructional  setting.  In  R.  F.  Dillon  &  J.  W.  Pellegrino  (Eds.),  Instruction:  Theoretical  and 
applied  perspectives.  New  York,  NY:  Praeger  Publishers. 

Morrison,  R.  F.,  &  Brantner,  T.  (in  preparation).  An  experience  based  learning  model:  A  pilot  study. 
San  Diego:  Navy  Personnel  Research  and  Development  Center. 

Vogt,  J.,  Gross,  S.  >•  Winnifeld,  M.  (1989).  Use  of  graphics  in  Navy  rate  training  manuals  (In 
preparation).  S.  Jiego:  Navy  Personnel  Research  and  Development  Center. 

Williams,  D.  (1989a).  ERPs  and  memory.  In  D.S.  Eggleston  (Ed.),  2nd  Navy  Independent 
Research! Independent  Exploratory  Development  Symposium:  Vol  1 .  203-219.  Laurel,  MD: 
Chemical  Propulsion  Information  Agency. 
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Independent  Research 

The  Cognitive  Electrophysiology  Project  (Williams  &  Lewis)  received  funding  from  the 
Office  of  Naval  Research  for  FY89  through  the  Navy  Laboratory  Participation  Program. 

This  research  was  nominated  for  the  best  independent  research  project  at  the  2nd  Navy 
Independent  Exploratory  Development  Symposium.  A  commendation  was  given  to  Drs.  Williams 
and  Lewis  by  the  Chief  of  Naval  Research  on  6  June  1988. 
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